305 Commits

Author SHA1 Message Date
zhucan
80ff4b90a5 e2e: test for node expand volume with secrets failed
Signed-off-by: zhucan <zhucan.k8s@gmail.com>
2023-03-14 21:12:31 +08:00
Kubernetes Prow Robot
6612af0af0 Merge pull request #110304 from RomanBednar/resize-e2e
add e2e test for correct resizing of a snaphot restore
2023-03-13 22:27:08 -07:00
Kubernetes Prow Robot
499a03d88b Merge pull request #115451 from zhucan/nodeexpandvolume-secret-e2e
e2e: add e2e test to node expand volume with secret
2023-03-08 07:53:12 -08:00
zhucan
7a87b4363b e2e: add e2e test to node expand volume with secret
Signed-off-by: zhucan <zhucan.k8s@gmail.com>
2023-03-08 10:27:50 +08:00
torredil
6aebda9b1e Remove AWS legacy cloud provider + EBS in-tree storage plugin
Signed-off-by: torredil <torredil@amazon.com>
2023-03-06 14:01:15 +00:00
Roman Bednar
7484c69a16 add e2e test for snapshot restore resizing 2023-02-27 10:47:02 +01:00
Jan Safranek
ba099644b2 Remove global framework variable
`f framework.Framework` does not need to be global, it's used only on a few
places.

This fixes vSphereDriver.PrepareTest() in in_tree.go that schedules
ginkgo.DeferCleanup() that uses the global `f` variable, but its value is not
valid at the time of ginkgo cleanup.
2023-02-20 11:00:12 +01:00
Patrick Ohly
136f89dfc5 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-02-06 15:39:13 +01:00
Antonio Ojea
7f5ae1c0c1 Revert "e2e: wait for pods with gomega" 2023-02-06 12:08:22 +01:00
Patrick Ohly
222f655062 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-01-31 13:01:39 +01:00
Patrick Ohly
bc6c7fa912 logging: fix names of keys
The stricter checking with the upcoming logcheck v0.4.1 pointed out these names
which don't comply with our recommendations in
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md#name-arguments.
2023-01-23 14:24:29 +01:00
Kubernetes Prow Robot
0d6dc14051 Merge pull request #114783 from pohly/e2e-framework-timeouts
e2e framework: consolidate timeouts and intervals
2023-01-12 03:29:08 -08:00
Hemant Kumar
58e7b97b49 Disable multiple pv mount tests for vsphere intree driver 2023-01-09 16:20:17 -05:00
Patrick Ohly
db394db398 e2e: move several timeouts from TestContext into TimeoutContext
This consolidates timeout handling. In the future, configuration of all
timeouts via a configuration file might get added. For now, the same three
legacy command line flags for the timeouts that get moved continue to be
supported.
2023-01-09 17:29:35 +01:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
2022-12-16 20:14:04 +01:00
Hemant Kumar
3d33178dab Start adding e2e for recovery from expansion failure 2022-12-14 15:28:05 -05:00
Patrick Ohly
d4729008ef e2e: simplify test cleanup
ginkgo.DeferCleanup has multiple advantages:
- The cleanup operation can get registered if and only if needed.
- No need to return a cleanup function that the caller must invoke.
- Automatically determines whether a context is needed, which will
  simplify the introduction of context parameters.
- Ginkgo's timeline shows when it executes the cleanup operation.
2022-12-13 08:09:01 +01:00
Patrick Ohly
8c27313cfd e2e storage: don't panic when server stops
Now that everything is connected to a per-test context, the gRPC server might
encounter an error before it gets shut down normally. We must not panic in that
case because it would kill the entire Ginkgo worker process. This is not even
an error, so just log it as info message.
2022-12-12 11:30:07 +01:00
Jan Safranek
6005c3b512 Remove OpenStack cloud provider
And keep OpenStack Cinder CSI translation
2022-11-11 12:04:55 +01:00
Jan Safranek
5b284a50b7 Revert "Remove references to openstack and cinder"
This reverts commit 9bbf01bae9.
2022-11-10 18:25:30 +01:00
Jan Safranek
d6c36736d5 Add mock CSI driver test for SELinux mount 2022-11-08 13:37:09 +01:00
Kubernetes Prow Robot
fc866d87dc Merge pull request #113208 from soltysh/azure_timeouts
Set default test timeouts first, only then modify the required ones
2022-10-20 12:07:20 -07:00
Kubernetes Prow Robot
0a08757181 Merge pull request #113128 from pohly/e2e-storage-cleanup
e2e storage: cleanup with context
2022-10-20 10:29:00 -07:00
Maciej Szulik
24d30bf8b2 Set default test timeouts first, only then modify the required ones 2022-10-20 15:35:17 +02:00
Patrick Ohly
d8d3dc9476 e2e storage: eliminate explicit test driver cleanup
Cleanup can be handled inside the setup code with ginkgo.DeferCleanup. This
simplifies the code.
2022-10-20 13:56:59 +02:00
Jan Safranek
bda42b6b53 Add capability for tests with multiple PVs with the same VolumeHandle
Some CSI drivers do not like multiple PVs with the same VolumeHandle,
therefore make that tests optional.
2022-10-18 13:44:54 +02:00
Kubernetes Prow Robot
216ba39632 Merge pull request #112987 from chrishenzie/e2e-test-rwop
e2e test for ReadWriteOncePod PV access mode
2022-10-17 10:01:08 -07:00
Chris Henzie
f6d9c27722 e2e test for ReadWriteOncePod PV access mode
Adds two tests for the enforcement of the ReadWriteOncePod
PersistentVolume access mode.

1. Tests that when two Pods are scheduled that reference the same
   ReadWriteOncePod PVC, the latter-scheduled Pod will be marked
   unschedulable because the PVC is in-use.
2. Tests that when two Pods are scheduled on the same node (setting
   Pod.Spec.NodeName to bypass scheduling for the second Pod), the
   latter Pod will fail to start because the PVC is already mounted on
   the Node.

Included are changes to update the hostpath CSI driver to accept new CSI
access modes. Its sidecar containers are already at supported versions
for ReadWriteOncePod and don't need updating. The GCP PD CSI driver does
not yet support the new CSI access modes, but its sidecar containers are
at supported versions and so the feature will work.

To support ReadWriteOncePod, the following CSI sidecars must be updated
to these versions or greater:
 - csi-provisioner:v3.0.0+
 - csi-attacher:v3.3.0+
 - csi-resizer:v1.3.0+

For more details, see:
https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md
2022-10-14 09:15:36 -07:00
Patrick Ohly
1793132198 e2e storage: avoid usage of stdin for file creation
It turned out to be unreliable (see
https://github.com/kubernetes/kubernetes/issues/112834).  Encoding the data
inside the command as input for base64 is a workaround that is fine for small
amounts of data. It becomes less efficient and/or unusable for large amounts.
2022-10-11 15:02:25 +02:00
Patrick Ohly
64731baffe e2e storage: add logging to proxy
This is optional and can be used to capture the result of command execution in
the log output.
2022-10-11 15:02:25 +02:00
Patrick Ohly
dfdf88d4fa e2e: adapt to moved code
This is the result of automatically editing source files like this:

    go install golang.org/x/tools/cmd/goimports@latest
    find ./test/e2e* -name "*.go" | xargs env PATH=$GOPATH/bin:$PATH ./e2e-framework-sed.sh

with e2e-framework-sed.sh containing this:

sed -i \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainer(/e2epod.ExecCommandInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainerWithFullOutput(/e2epod.ExecCommandInContainerWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInContainer(/e2epod.ExecShellInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPod(/e2epod.ExecShellInPod(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPodWithFullOutput(/e2epod.ExecShellInPodWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecWithOptions(/e2epod.ExecWithOptions(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.MatchContainerOutput(/e2eoutput.MatchContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClient(/e2epod.NewPodClient(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClientNS(/e2epod.PodClientNS(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutput(/e2eoutput.TestContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutputRegexp(/e2eoutput.TestContainerOutputRegexp(\1, /" \
    -e "s/framework.AddOrUpdateLabelOnNode\b/e2enode.AddOrUpdateLabelOnNode/" \
    -e "s/framework.AllNodes\b/e2edebug.AllNodes/" \
    -e "s/framework.AllNodesReady\b/e2enode.AllNodesReady/" \
    -e "s/framework.ContainerResourceGatherer\b/e2edebug.ContainerResourceGatherer/" \
    -e "s/framework.ContainerResourceUsage\b/e2edebug.ContainerResourceUsage/" \
    -e "s/framework.CreateEmptyFileOnPod\b/e2eoutput.CreateEmptyFileOnPod/" \
    -e "s/framework.DefaultPodDeletionTimeout\b/e2epod.DefaultPodDeletionTimeout/" \
    -e "s/framework.DumpAllNamespaceInfo\b/e2edebug.DumpAllNamespaceInfo/" \
    -e "s/framework.DumpDebugInfo\b/e2eoutput.DumpDebugInfo/" \
    -e "s/framework.DumpNodeDebugInfo\b/e2edebug.DumpNodeDebugInfo/" \
    -e "s/framework.EtcdUpgrade\b/e2eproviders.EtcdUpgrade/" \
    -e "s/framework.EventsLister\b/e2edebug.EventsLister/" \
    -e "s/framework.ExecOptions\b/e2epod.ExecOptions/" \
    -e "s/framework.ExpectNodeHasLabel\b/e2enode.ExpectNodeHasLabel/" \
    -e "s/framework.ExpectNodeHasTaint\b/e2enode.ExpectNodeHasTaint/" \
    -e "s/framework.GCEUpgradeScript\b/e2eproviders.GCEUpgradeScript/" \
    -e "s/framework.ImagePrePullList\b/e2epod.ImagePrePullList/" \
    -e "s/framework.KubectlBuilder\b/e2ekubectl.KubectlBuilder/" \
    -e "s/framework.LocationParamGKE\b/e2eproviders.LocationParamGKE/" \
    -e "s/framework.LogSizeDataTimeseries\b/e2edebug.LogSizeDataTimeseries/" \
    -e "s/framework.LogSizeGatherer\b/e2edebug.LogSizeGatherer/" \
    -e "s/framework.LogsSizeData\b/e2edebug.LogsSizeData/" \
    -e "s/framework.LogsSizeDataSummary\b/e2edebug.LogsSizeDataSummary/" \
    -e "s/framework.LogsSizeVerifier\b/e2edebug.LogsSizeVerifier/" \
    -e "s/framework.LookForStringInLog\b/e2eoutput.LookForStringInLog/" \
    -e "s/framework.LookForStringInPodExec\b/e2eoutput.LookForStringInPodExec/" \
    -e "s/framework.LookForStringInPodExecToContainer\b/e2eoutput.LookForStringInPodExecToContainer/" \
    -e "s/framework.MasterAndDNSNodes\b/e2edebug.MasterAndDNSNodes/" \
    -e "s/framework.MasterNodes\b/e2edebug.MasterNodes/" \
    -e "s/framework.MasterUpgradeGKE\b/e2eproviders.MasterUpgradeGKE/" \
    -e "s/framework.NewKubectlCommand\b/e2ekubectl.NewKubectlCommand/" \
    -e "s/framework.NewLogsVerifier\b/e2edebug.NewLogsVerifier/" \
    -e "s/framework.NewNodeKiller\b/e2enode.NewNodeKiller/" \
    -e "s/framework.NewResourceUsageGatherer\b/e2edebug.NewResourceUsageGatherer/" \
    -e "s/framework.NodeHasTaint\b/e2enode.NodeHasTaint/" \
    -e "s/framework.NodeKiller\b/e2enode.NodeKiller/" \
    -e "s/framework.NodesSet\b/e2edebug.NodesSet/" \
    -e "s/framework.PodClient\b/e2epod.PodClient/" \
    -e "s/framework.RemoveLabelOffNode\b/e2enode.RemoveLabelOffNode/" \
    -e "s/framework.ResourceConstraint\b/e2edebug.ResourceConstraint/" \
    -e "s/framework.ResourceGathererOptions\b/e2edebug.ResourceGathererOptions/" \
    -e "s/framework.ResourceUsagePerContainer\b/e2edebug.ResourceUsagePerContainer/" \
    -e "s/framework.ResourceUsageSummary\b/e2edebug.ResourceUsageSummary/" \
    -e "s/framework.RunHostCmd\b/e2eoutput.RunHostCmd/" \
    -e "s/framework.RunHostCmdOrDie\b/e2eoutput.RunHostCmdOrDie/" \
    -e "s/framework.RunHostCmdWithFullOutput\b/e2eoutput.RunHostCmdWithFullOutput/" \
    -e "s/framework.RunHostCmdWithRetries\b/e2eoutput.RunHostCmdWithRetries/" \
    -e "s/framework.RunKubectl\b/e2ekubectl.RunKubectl/" \
    -e "s/framework.RunKubectlInput\b/e2ekubectl.RunKubectlInput/" \
    -e "s/framework.RunKubectlOrDie\b/e2ekubectl.RunKubectlOrDie/" \
    -e "s/framework.RunKubectlOrDieInput\b/e2ekubectl.RunKubectlOrDieInput/" \
    -e "s/framework.RunKubectlWithFullOutput\b/e2ekubectl.RunKubectlWithFullOutput/" \
    -e "s/framework.RunKubemciCmd\b/e2ekubectl.RunKubemciCmd/" \
    -e "s/framework.RunKubemciWithKubeconfig\b/e2ekubectl.RunKubemciWithKubeconfig/" \
    -e "s/framework.SingleContainerSummary\b/e2edebug.SingleContainerSummary/" \
    -e "s/framework.SingleLogSummary\b/e2edebug.SingleLogSummary/" \
    -e "s/framework.TimestampedSize\b/e2edebug.TimestampedSize/" \
    -e "s/framework.WaitForAllNodesSchedulable\b/e2enode.WaitForAllNodesSchedulable/" \
    -e "s/framework.WaitForSSHTunnels\b/e2enode.WaitForSSHTunnels/" \
    -e "s/framework.WorkItem\b/e2edebug.WorkItem/" \
    "$@"

for i in "$@"; do
    # Import all sub packages and let goimports figure out which of those
    # are redundant (= already imported) or not needed.
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2edebug "k8s.io/kubernetes/test/e2e/framework/debug"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2ekubectl "k8s.io/kubernetes/test/e2e/framework/kubectl"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2enode "k8s.io/kubernetes/test/e2e/framework/node"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eoutput "k8s.io/kubernetes/test/e2e/framework/pod/output"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2epod "k8s.io/kubernetes/test/e2e/framework/pod"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eproviders "k8s.io/kubernetes/test/e2e/framework/providers"' "$i"
    goimports -w "$i"
done
2022-10-06 08:19:47 +02:00
Patrick Ohly
5b63199f46 e2e storage: add Rename to PodIO
This is useful for atomic file creation: first create a temporary file, then
rename it.
2022-09-30 13:25:26 +02:00
Humble Chirammal
9e9fc2be88 various corrections in test/e2e package
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 18:59:30 +05:30
Humble Chirammal
c2a7bfd6b4 removal of glusterfs in-tree driver code from the source
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-13 20:11:50 +05:30
Kubernetes Prow Robot
eedf0ed221 Merge pull request #112390 from pohly/e2e-storage-helpers
e2e storage: update helpers
2022-09-12 08:07:48 -07:00
Kubernetes Prow Robot
48256c55c2 Merge pull request #112251 from pohly/e2e-storage-proxy-shutdown
e2e storage: close all pending streams when proxy listener gets closed
2022-09-12 08:07:24 -07:00
Patrick Ohly
b6b25c80de e2e storage: add CreateFile to pod IO operations
This complements the other operations and will be used by some
future tests.
2022-09-12 14:38:27 +02:00
Patrick Ohly
fc092eb145 e2e: remove cleanup actions
The framework.AddCleanupAction API was a workaround for Ginkgo v1 not invoking
AfterEach callbacks after a test failure. Ginkgo v2 not only fixed that, but
also added a DeferCleanup API which can be used to run some code if (and only
if!) the corresponding setup code ran. In several cases that makes the test
cleanup simpler.
2022-09-09 13:51:07 +02:00
Patrick Ohly
8349c77e82 e2e storage: better explanation for maxConcurrentConnections
This the same behavior as before, it just wasn't documented.
2022-09-07 17:05:22 +02:00
Patrick Ohly
0bd9a1a29a e2e storage: close all pending streams when proxy listener gets closed
After updating gRPC in node-driver-registrar from v1.40.0 to v1.47.0 the
behavior of gRPC change in a way such that it no longer detected the
single-sided closing of the stream as a loss of connection. This caused gRPC in
the e2e.test to get stuck, possibly in a Read or Write for the HTTP stream
because those have neither a context nor a timeout.

Changing the connection handling so that all active connections are tracking in
the listener and closing them when the listener gets closed fixed this problem.
2022-09-05 22:46:01 +02:00
Joe Betz
7be77f9ecc changes code to improve output for files under test/e2e/storage 2022-09-02 17:43:41 +05:30
S R Ashrith
41dbdef540 vsphere in-tree driver test fixes 2022-08-27 23:14:40 +05:30
Davanum Srinivas
9bbf01bae9 Remove references to openstack and cinder
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-08-22 16:43:14 -04:00
Kubernetes Prow Robot
eedd2b7cb4 Merge pull request #108696 from wongma7/zonepreprovisioned
Set zone label selector for preprovisioned AWS in-tree tests
2022-08-01 17:46:28 -07:00
Abirdcfly
00b9ead02c cleanup: remove duplicate import
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-14 11:25:19 +08:00
Fabio Bertinatto
eefa0fd2ed e2e: bump Azure Disk in-tree timeouts
Some operations with Azure Disk volumes can be arbitrarily slow
sometimes. This causes CI flakes that are hard to debug.
Bumping the timeouts has sown to improve or eliminate this issue.
2022-07-08 11:46:26 -03:00
Dave Chen
857458cfa5 update ginkgo from v1 to v2 and gomega to 1.19.0
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:46 +08:00
Jonathan Dobson
c8d3cc5c07 e2e: restore volume lifecycle checks for csi-hostpath driver
These tests were previously disabled to work around #79980
https://github.com/kubernetes/kubernetes/commit/f1e1f3a416b70bafadf961518c330ce3b1b5459
2022-06-01 14:23:10 -06:00
xin.li
ba654e0dad modify link
Signed-off-by: xin.li <xin.li@daocloud.io>
2022-05-05 00:19:02 +08:00
Jonathan Dobson
f369b1234a e2e: add storage capability for offline volume expansion 2022-04-21 14:49:18 -06:00