Commit Graph

862 Commits

Author SHA1 Message Date
Stephen Kitt
beb51e1759 make verify: run checks in all module hack directories
This ensures that all verification scripts are run throughout the
repository.

Signed-off-by: Stephen Kitt <skitt@redhat.com>
2024-09-26 08:46:04 +02:00
Tim Hockin
8912df652b Use Go workspaces + go list to find test targets
Plain old UNIX find requires us to do all sorts of silly filtering.
2024-09-24 09:04:13 -07:00
Tim Hockin
7d89e9b4c0 Only normalize user-provided test targets 2024-09-23 16:25:29 -07:00
Achraf BOUAOUDA
e645e06a7f fix(e2e-node-tests): project and zone in the prerequisites check 2024-09-19 12:23:08 +02:00
Patrick Ohly
a7da865aa8 test: filter "go test" output with gotestsum instead of grep
Filtering the output with grep leads to hard to read log output, e.g. from
pull-kubernetes-unit:

    +++ [0613 15:32:48] Running tests without code coverage and with -race
    {"Time":"2024-06-13T15:33:47.845457374Z","Action":"output","Package":"k8s.io/kubernetes/cluster/gce/cos","Test":"TestCreateMasterAuditPolicy","Output":"        /tmp/configure-helper-test47992121/kube-env: line 1: `}'\n"}
    {"Time":"2024-06-13T15:33:49.053732803Z","Action":"output","Package":"k8s.io/kubernetes/cluster/gce/cos","Output":"ok  \tk8s.io/kubernetes/cluster/gce/cos\t2.906s\n"}

We can do better than that. When feeding the output of the "go test" command(s)
into gotestsum *while it runs*, we can use --format=standard-quiet (= normal go
test output) or --format=standard-verbose (= `go test -v`) when FULL_LOG is
requested to get nicer output.

This works when testing everything at once. This was said to be not possible
when doing coverage profiling. But recent Go no longer has that limitation, so
the xargs trick gets removed. All that we need to do for coverage profiling is
to add some additional parameters and the conversion to HTML.
2024-08-19 16:14:05 +02:00
Kubernetes Prow Robot
69dbf2eee9 Merge pull request #126665 from liggitt/version-build-id
Restore honoring --version build ID overrides
2024-08-13 22:12:17 -07:00
Jordan Liggitt
c181912dc5 Restore honoring --version build ID overrides 2024-08-13 21:52:58 -04:00
Siyuan Zhang
35488ef5c7 Verify: add static analysis to verify new feature gates are added as versioned feature specs.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-08-01 13:52:32 -07:00
Kubernetes Prow Robot
3d24b962be Merge pull request #124505 from jsafrane/clean-pvlabeler
Remove PersistentVolumeLabel admission plugin
2024-05-09 08:52:48 -07:00
Jan Safranek
e7a6ed2e3d Remove PersistentVolumeLabel admission plugin
Remove useless admission plugin.

* It has been deprecated for years.
* All in-tree cloud providers were removed, so the admission plugin does not have
  any way to get PV labels.
* There is a replacement in https://github.com/kubernetes-sigs/cloud-pv-admission-labeler
2024-05-09 11:10:14 +02:00
Davanum Srinivas
fa7cdf81cc Drop typecheck-providerless verification
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2024-05-07 08:24:15 -04:00
Kubernetes Prow Robot
d8f8c7fae0 Merge pull request #124288 from pohly/test-e2e-node-debugger
e2e node: debugger support
2024-04-22 08:43:27 -07:00
Patrick Ohly
d97b67d97a e2e node: support running the test binary under a debugger
Single-stepping interactively through a test can be useful to understand what's
happening and to investigate the state at each step.

Similar support was added early to hack/ginkgo-e2e.sh, so the same env variable
is used again.
2024-04-16 11:46:28 +02:00
Ed Bartosh
91099aca72 hacks: build main target without go in the PATH
`make` is able to build project binaries, but fails with
error `hack/lib/golang.sh: line 455: go: command not found`
trying to place them if go binary is not in the PATH.
This happens because kube::golang::place_bins uses different
environment than kube::golang::build_binaries.

Setting up an one environment for both `kube::golang::place_bins`
and `kube::golang::build_binaries` should solve this issue and allow
default make target to fully work without go binary in the PATH.
2024-04-05 17:36:57 +03:00
Tim Hockin
a0215badb8 Change pushd $d; go ...; popd to go -C $d ...
Just an annoyance in reading these scripts.
2024-03-02 14:40:10 -08:00
Tim Hockin
5a96a56c80 Fix make test-integration for workspaces 2024-02-29 22:06:57 -08:00
Tim Hockin
e84adf611f Remove references to "GO111MODULE" 2024-02-29 22:06:55 -08:00
Tim Hockin
d772f7719d Remove defunct references to "vendor" 2024-02-29 22:06:52 -08:00
Tim Hockin
10c32b3e2f Get rid of most references to GOPATH 2024-02-29 22:06:51 -08:00
Tim Hockin
530c4352fb Rename new::setup_env back to setup_env 2024-02-29 22:06:44 -08:00
Tim Hockin
c79e5abec4 Change test-e2e-node to use new setup_env 2024-02-29 22:06:17 -08:00
Tim Hockin
5f490b4fe8 Clarify, document KUBE_OUTPUT_* variables 2024-02-29 22:05:42 -08:00
Tim Hockin
f3c8e92def Clean up test.sh and get rid of GOPATH
Result: `make test` works:

```
$ #### BUILD ####

$ make WHAT="./cmd/kubectl/ ./staging/src/k8s.io/api"
+++ [1211 11:20:46] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)
    ./staging/src/k8s.io/api (non-static)

$ #### TEST ####

$ make test WHAT=./pkg/proxy/iptables/
+++ [1211 11:22:31] Set GOMAXPROCS automatically to 6
+++ [1211 11:22:31] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	9.517s

$ make test WHAT=pkg/proxy/iptables/
+++ [1211 11:22:53] Set GOMAXPROCS automatically to 6
+++ [1211 11:22:53] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	(cached)

$ make test WHAT=k8s.io/kubernetes/pkg/proxy/iptables/
+++ [1211 11:23:09] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:09] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	(cached)

$ make test WHAT=./staging/src/k8s.io/api
+++ [1211 11:23:24] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:24] Running tests without code coverage and with -race
ok  	k8s.io/api	21.981s

$ make test WHAT=staging/src/k8s.io/api
+++ [1211 11:23:54] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:54] Running tests without code coverage and with -race
ok  	k8s.io/api	(cached)

$ make test WHAT=k8s.io/api
+++ [1211 11:24:06] Set GOMAXPROCS automatically to 6
+++ [1211 11:24:06] Running tests without code coverage and with -race
ok  	k8s.io/api	(cached)
```
2024-02-29 00:22:13 -08:00
Tim Hockin
81d932906a Make golang::binaries_from_targets smarter
and use it from test

Result: `make` works, but `make test` still broken

```
$ make kubectl
+++ [1211 11:15:46] Building go targets for linux/amd64
    ./cmd/kubectl (static)

$ make WHAT=./cmd/kubectl/
+++ [1211 11:15:53] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)

$ make WHAT=cmd/kubectl/
+++ [1211 11:16:02] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)

$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:16:16] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)

$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:16:31] Building go targets for linux/amd64
    ./staging/src/k8s.io/api (non-static)

$ make WHAT=staging/src/k8s.io/api
+++ [1211 11:16:43] Building go targets for linux/amd64
    ./staging/src/k8s.io/api (non-static)

$ make WHAT=k8s.io/api
+++ [1211 11:16:50] Building go targets for linux/amd64
    k8s.io/api (non-static)
```

```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:17:23] Set GOMAXPROCS automatically to 6
+++ [1211 11:17:23] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
2024-02-29 00:22:08 -08:00
Tim Hockin
8b579b2347 Change the build_binaries path to use modules
This makes "new" and "old" setup_env functions. In subsequent commits,
all callers of the "old" form will be fixed, and the "new" will be
renamed back.

The old and new functions diff:

```diff
--- /tmp/a	2023-12-14 09:02:57.804092696 -0800
+++ /tmp/b	2023-12-14 09:03:09.679999585 -0800
@@ -1,4 +1,4 @@
-kube::golang::old::setup_env() {
+kube::golang:🆕:setup_env() {
   kube::golang::verify_go_version

   # Set up GOPATH.  We have tools which depend on being in a GOPATH (see
@@ -7,9 +7,9 @@
   # Even in module mode, we need to set GOPATH for `go build` and `go install`
   # to work.  We build various tools (usually via `go install`) from a lot of
   # scripts.
-  #   * We can't set GOBIN because that does not work on cross-compiles.
-  #   * We could use `go build -o <something>`, but it's subtle when it comes
-  #     to cross-compiles and whether the <something> is a file or a directory,
+  #   * We can't just set GOBIN because that does not work on cross-compiles.
+  #   * We could always use `go build -o <something>`, but it's subtle wrt
+  #     cross-compiles and whether the <something> is a file or a directory,
   #     and EVERY caller has to get it *just* right.
   #   * We could leave GOPATH alone and let `go install` write binaries
   #     wherever the user's GOPATH says (or doesn't say).
@@ -20,16 +20,6 @@
   #
   # Eventually, when we no longer rely on run-in-gopath.sh we may be able to
   # simplify this some.
-  local go_pkg_dir="${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}"
-  local go_pkg_basedir
-  go_pkg_basedir=$(dirname "${go_pkg_dir}")
-
-  mkdir -p "${go_pkg_basedir}"
-
-  # TODO: This symlink should be relative.
-  if [[ ! -e "${go_pkg_dir}" || "$(readlink "${go_pkg_dir}")" != "${KUBE_ROOT}" ]]; then
-    ln -snf "${KUBE_ROOT}" "${go_pkg_dir}"
-  fi
   export GOPATH="${KUBE_GOPATH}"

   # If these are not set, set them now.  This ensures that any subsequent
@@ -40,24 +30,10 @@
   # Make sure our own Go binaries are in PATH.
   export PATH="${KUBE_GOPATH}/bin:${PATH}"

-  # Change directories so that we are within the GOPATH.  Some tools get really
-  # upset if this is not true.  We use a whole fake GOPATH here to collect the
-  # resultant binaries.
-  local subdir
-  subdir=$(kube::realpath . | sed "s|${KUBE_ROOT}||")
-  cd "${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}/${subdir}" || return 1
-
-  # Set GOROOT so binaries that parse code can work properly.
-  GOROOT=$(go env GOROOT)
-  export GOROOT
-
   # Unset GOBIN in case it already exists in the current session.
   # Cross-compiles will not work with it set.
   unset GOBIN

-  # This seems to matter to some tools
-  export GO15VENDOREXPERIMENT=1
-
-  # Disable workspaces
-  export GOWORK=off
+  # Explicitly turn on modules.
+  export GO111MODULE=on
 }
```

Result: `make` works for k/k:

```
$ make kubectl
+++ [1211 11:07:31] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)

$ make WHAT=./cmd/kubectl/
+++ [1211 11:08:19] Building go targets for linux/amd64
    k8s.io/kubernetes/./cmd/kubectl/ (non-static)

$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:08:52] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)
```

Result: `make` works for staging by package:

```
$ make WHAT=k8s.io/api
+++ [1211 11:11:37] Building go targets for linux/amd64
    k8s.io/api (non-static)
```

Result: `make` fails for staging by path:

```
$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:12:44] Building go targets for linux/amd64
    k8s.io/kubernetes/./staging/src/k8s.io/api (non-static)
cannot find module providing package k8s.io/kubernetes/staging/src/k8s.io/api: import lookup disabled by -mod=vendor
	(Go version in go.work is at least 1.14 and vendor directory exists.)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: /home/thockin/src/kubernetes/hack/lib/golang.sh:850 kube::golang::build_some_binaries(...)
!!! [1211 11:12:44]  2: /home/thockin/src/kubernetes/hack/lib/golang.sh:1012 kube::golang::build_binaries_for_platform(...)
!!! [1211 11:12:44]  3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```

Result: `make test` fails:

```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:13:38] Set GOMAXPROCS automatically to 6
+++ [1211 11:13:38] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
2024-02-29 00:22:07 -08:00
Patrick Ohly
65b841c077 Generate go.work files
This creates go.work and enables Go Workspaces.  This is a file that
includes info on k/k and all the staging modules.

This depends on go 1.22 and setting FORCE_HOST_GO=true (for kube
scripts, which try to be hermetic).

Make this part of the normal update/verify sequence.

The top-level go.work file contains no replace statements. Instead, the
replace statements in the individual go.mod files are used. For this to
work, replace statements in the individual go.mod files have to be
consistent.

hack/tools has different dependencies and can't be in the main
workspace, so this adds a go.work just for that.  Without this, go tries
to consider all deps in all modules and pick one that works for all.
This is problematic because there are so many of them that it is
difficult to manage.

Likewise for k8s.io/code-generator/examples and
k8s.io/kms/internal/plugins/_mock - add trivial go.work files.

For example k/k depends on an older version of a lib that gloangci-lint
needs (transitively) and it breaks.

This also updates vendor (needed to make go happy), and removes
vendor'ed symlinks.  This breaks a LOT of our build tools, which will be
fixed subsequently.

Result: `go` commands work across modules:

Before:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api
```

After:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
k8s.io/kubernetes/pkg/proxy/iptables
k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
ok  	k8s.io/kubernetes/pkg/proxy/iptables	0.360s
ok  	k8s.io/api	2.302s
```

Result: `make` fails:

```
$ make
go version go1.22rc1 linux/amd64
+++ [0106 12:11:03] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kube-proxy (static)
    k8s.io/kubernetes/cmd/kube-apiserver (static)
    k8s.io/kubernetes/cmd/kube-controller-manager (static)
    k8s.io/kubernetes/cmd/kubelet (non-static)
    k8s.io/kubernetes/cmd/kubeadm (static)
    k8s.io/kubernetes/cmd/kube-scheduler (static)
    k8s.io/component-base/logs/kube-log-runner (static)
    k8s.io/kube-aggregator (static)
    k8s.io/apiextensions-apiserver (static)
    k8s.io/kubernetes/cluster/gce/gci/mounter (static)
    k8s.io/kubernetes/cmd/kubectl (static)
    k8s.io/kubernetes/cmd/kubectl-convert (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e/e2e.test (test)
    k8s.io/kubernetes/test/conformance/image/go-runner (non-static)
    k8s.io/kubernetes/cmd/kubemark (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e_node/e2e_node.test (test)
test/e2e/e2e.go:35:2: cannot find package "k8s.io/api/apps/v1" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/api/apps/v1 (vendor tree)
	/home/thockin/src/kubernetes/_output/local/.gimme/versions/go1.22rc1.linux.amd64/src/k8s.io/api/apps/v1 (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/api/apps/v1 (from $GOPATH)
	... more ...
	... more ...
	... more ...
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: /home/thockin/src/kubernetes/hack/lib/golang.sh:948 kube::golang::build_binaries_for_platform(...)
!!! [0106 12:13:41]  2: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```

Again, this requires go 1.22 (e.g. gotip), as go 1.21.x does not have
`go work vendor` support.

TO REPEAT:
    ( \
      ./hack/update-go-workspace.sh; \
      ./hack/update-vendor.sh; \
      ./hack/update-go-workspace.sh; \
    )
2024-02-29 00:22:06 -08:00
Christoph Blecker
0a1edf8bda Re-add KUBE_TIMEOUT explicitly to make test 2024-02-26 15:46:38 -08:00
Christoph Blecker
ca41a9222d Fix makefile variable expansion for test-integration target 2024-02-26 09:01:04 -08:00
Patrick Ohly
c1f2fe7bcb golangci-lint: optionally skip it during "make verify", II
The pull-kubernetes-verify job is using this to run the base
verify-golangci.sh only in the pull-kubernetes-verify-lint job.
Because the file name was not quite right, it still ran.
2024-02-16 10:07:51 +01:00
Patrick Ohly
39606af8c3 golangci-lint: optionally skip it during "make verify"
The pull-kubernetes-verify job is going to use this to run the base
verify-golangci.sh only in the pull-kubernetes-verify-lint job.
2024-02-13 17:35:15 +01:00
Kubernetes Prow Robot
c0618cf947 Merge pull request #122769 from thockin/bump_shellcheck_version
Bump shellcheck to 0.9.0
2024-02-05 08:45:29 -08:00
Tim Hockin
a7cf7fe55a Bump shellcheck to 0.9.0
Fix everything it flagged.
2024-01-26 16:26:40 -08:00
bells17
62eb7cb4f9 make update: add update-golangci-lint-config 2024-01-23 03:09:13 +09:00
Tim Hockin
8b8f0a70cd Make verify-fieldname-docs actually error
Sadly, cmd/fieldnamedocscheck did not exit non-zero when it needed to.

Fix the one thing it flagged.
Add it to verify-quick
2024-01-14 10:32:08 -08:00
Kubernetes Prow Robot
88d497d401 Merge pull request #118626 from borg-land/use-e2-instance-types
Switch to e2 machine types
2024-01-04 16:49:14 +01:00
Tim Hockin
b9ff6b25f2 Simpler and less noisy make-clean 2023-12-08 09:20:44 -08:00
upodroid
e2227a24fb switch to e2 machine types 2023-10-31 21:46:14 +00:00
Patrick Ohly
f538be659c hack: fix update of golangci-lint verify scripts
b190ea0c9 accidentally enabled verify-golangci-lint-pr-hints.sh (non-blocking!)
in the normal "make verify" (blocking!).
2023-10-10 09:03:20 +02:00
Patrick Ohly
b190ea0c96 hack: update golangci-lint verify scripts
Instead of invoking verify-golangci-lint.sh directly from Prow jobs,
those Prow jobs should use "make verify WHAT=...". The advantage is
that the common code for running verify targets will be used, which
includes producing JUnit files.

Providing simple wrappers for strict linting of PRs (=
verify-golangci-lint-pr.sh) and event stricter linting of PRs with hints
enabled (= verify-golangci-lint-pr-hints.sh) enables those WHAT targets.
2023-10-09 20:14:47 +02:00
Jordan Liggitt
bd36be0218 Make gomaxprocs install optional, limit to tests 2023-08-16 09:33:02 -04:00
upodroid
1c99f9591b add node-env and instance-type flags to node-e2e tests 2023-07-21 21:46:37 +00:00
Davanum Srinivas
564c236ce2 Do NOT prepull images for e2e-node jobs
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2023-06-19 22:01:12 -04:00
Davanum Srinivas
89adbc6e5b check for AWS environment before running sudo
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2023-06-14 14:03:44 -04:00
Dave Chen
54cae70678 Pass the mandatory parameters to support testing on ARM64
The parameters are needed to build the test binaries, e.g. `e2e_node.test`
`ginkgo` etc.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2023-06-08 17:33:37 +08:00
Kubernetes Prow Robot
f812ac5445 Merge pull request #118349 from bart0sh/PR116-fix-ginkgo-deprecation-warning
Fix ginkgo deprecation warning
2023-06-06 06:59:01 -07:00
Patrick Ohly
dbbb21469f verify: pick relevant lines from verify-golangci-lint.sh as failure message
When sh2ju.sh was called to generate the junit_verify.xml, it used to include
the entire output of a failed script twice: once as failure message, once as
log output.

This output can be large and often the actual failure isn't near the top, but
rather at the end or (in the case of the different golangci-lint invocations)
embedded in the log. This makes them hard to see at a glance when looking at
the Prow result page for a job.

Now a verify script can prefix relevant lines with "ERROR: " and then only
those lines are used as failure message in JUnit, without that prefix.

That string was chosen because Prow itself also then picks up those lines when
viewing the entire build log and it is unlikely that some script prints such
lines when they are not meant to be part of the failure.

If some script outputs no such lines, "see stderr for details" is used as
failure message. This is better than before because it avoids the redundancy.
2023-06-02 15:37:50 +02:00
Ed Bartosh
c48c4c679a Fix ginkgo deprecation warning
Fixed ginkgo warning
   You're using deprecated Ginkgo functionality:
   =============================================
   --untilItFails is deprecated, use --until-it-fails instead

Used consistent approach with this flag in e2e_node and e2e scripts.
2023-05-31 12:53:11 +03:00
Patrick Ohly
d459b4429f integration: remove special verbosity for garbagecollector and graph_builder
The setting for garbagecollector was added 7 years ago in 9ac91e5172 for
"debugging gc".  graph_builder was added 6 years in a98801c1 when restoring the
-vmodule parameter after some temporary removal, without an explanation.

It seems safe to assume that the garbage collector has been debugged
sufficiently...

These defaults cause performance overhead:
- Enabling -vmodule slows down all log calls because checking verbosity
  cannot take a simpler fast path.
- The amount of log output is much higher for those files.

The amount of log data also caused test output to get truncated, removing the
actual test failure explanation.
2023-05-24 10:02:53 +02:00
Tim Allclair
3808920328 Add a script to verify that URLs in API documentation are valid 2023-03-13 11:37:59 -07:00
Patrick Ohly
04700e97d3 hack: prepare for strict golangci-lint in pull-verify-kubernetes
The long-term goal is that when "make verify" is invoked in pull job, it will
also run golangci-lint with the strict configuration and write an
$ARTIFACTS/golangci-lint-githubactions.log file with GitHub actions
annotations. How to get those published for the GitHub PR is open.

When "make verify" is invoked manually or in any other job, the stricter check
will be skipped. That works because "PR_NUMBER" is only set for pre-merge
jobs (https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables).

Because strict linting is still new and might not be useful without a better
UI (= GitHub annotations), this PR does not fully enable the integration into
"make verify". Instead, the new verify-golangci-lint-pr.sh is excluded from it
and needs to be run in a separate job.
2023-03-07 19:38:32 +01:00