Commit Graph

154 Commits

Author SHA1 Message Date
liyuerich
f55588fa0b e2e_storage:stop using deprecated framework.ExpectError
Signed-off-by: liyuerich <yue.li@daocloud.io>
2024-05-09 18:10:33 +08:00
carlory
2bd7f4f8e0 storage e2e: update hostpath and mock images 2024-03-05 10:18:54 +08:00
Patrick Ohly
f2d34426f8 e2e: enhance SIGDescribe
framework.SIGDescribe is better because:
- Ginkgo uses the source code location of the test, not of the wrapper,
  when reporting progress.
- Additional annotations can be passed.

To make this a drop-in replacement, framework.SIGDescribe generates a function
that can be used instead of the former SIGDescribe functions.

windows.SIGDescribe contained some additional code to ensure that tests are
skipped when not running with a suitable node OS. This gets moved into a
separate wrapper generator, to allow using framework.SIGDescribe as intended.
To ensure that all callers were modified, the windows.sigDescribe isn't
exported anymore (wasn't necessary in the first place!).
2023-10-10 18:15:49 +02:00
宋文杰
8a368afa27 e2e_storage: stop using deprecated framework.ExpectEqual 2023-08-30 14:00:34 +08:00
Davanum Srinivas
90d185b7e1 Drop AWS kubelet credential provider and cleanup AWS storage e2e tests
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2023-03-07 09:00:12 -05:00
Patrick Ohly
136f89dfc5 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-02-06 15:39:13 +01:00
Antonio Ojea
7f5ae1c0c1 Revert "e2e: wait for pods with gomega" 2023-02-06 12:08:22 +01:00
Patrick Ohly
222f655062 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-01-31 13:01:39 +01:00
Kubernetes Prow Robot
3244ebd532 Merge pull request #113973 from my-git9/invalidurl
fix invalid url
2023-01-30 18:42:48 -08:00
David Porter
6cb022ff4a test: Update Host Exec to support node e2e
Node E2E tests do not run a scheduler, so the host exec pod must have
the `spec.nodeName` set explicitly.

Signed-off-by: David Porter <david@porter.me>
2023-01-26 00:37:01 -08:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
2022-12-16 20:14:04 +01:00
Hemant Kumar
3d33178dab Start adding e2e for recovery from expansion failure 2022-12-14 15:28:05 -05:00
Patrick Ohly
d4729008ef e2e: simplify test cleanup
ginkgo.DeferCleanup has multiple advantages:
- The cleanup operation can get registered if and only if needed.
- No need to return a cleanup function that the caller must invoke.
- Automatically determines whether a context is needed, which will
  simplify the introduction of context parameters.
- Ginkgo's timeline shows when it executes the cleanup operation.
2022-12-13 08:09:01 +01:00
xin.li
5aeb9db52f fix invalid url
Signed-off-by: xin.li <xin.li@daocloud.io>
2022-11-17 18:24:54 +08:00
Jan Safranek
3fdd4ca7a4 Fix subpath disruptive tests
Add volumePath parameter to all disruptive checks, so subpath tests can use
"/test-volume" and disruptive tests can use "/mnt/volume1" for their
respective Pods.
2022-11-11 14:03:33 +01:00
Jan Safranek
d6c36736d5 Add mock CSI driver test for SELinux mount 2022-11-08 13:37:09 +01:00
Jan Safranek
802979c295 Add SELinux disruptive test 2022-11-08 12:42:20 +01:00
Patrick Ohly
dfdf88d4fa e2e: adapt to moved code
This is the result of automatically editing source files like this:

    go install golang.org/x/tools/cmd/goimports@latest
    find ./test/e2e* -name "*.go" | xargs env PATH=$GOPATH/bin:$PATH ./e2e-framework-sed.sh

with e2e-framework-sed.sh containing this:

sed -i \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainer(/e2epod.ExecCommandInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainerWithFullOutput(/e2epod.ExecCommandInContainerWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInContainer(/e2epod.ExecShellInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPod(/e2epod.ExecShellInPod(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPodWithFullOutput(/e2epod.ExecShellInPodWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecWithOptions(/e2epod.ExecWithOptions(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.MatchContainerOutput(/e2eoutput.MatchContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClient(/e2epod.NewPodClient(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClientNS(/e2epod.PodClientNS(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutput(/e2eoutput.TestContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutputRegexp(/e2eoutput.TestContainerOutputRegexp(\1, /" \
    -e "s/framework.AddOrUpdateLabelOnNode\b/e2enode.AddOrUpdateLabelOnNode/" \
    -e "s/framework.AllNodes\b/e2edebug.AllNodes/" \
    -e "s/framework.AllNodesReady\b/e2enode.AllNodesReady/" \
    -e "s/framework.ContainerResourceGatherer\b/e2edebug.ContainerResourceGatherer/" \
    -e "s/framework.ContainerResourceUsage\b/e2edebug.ContainerResourceUsage/" \
    -e "s/framework.CreateEmptyFileOnPod\b/e2eoutput.CreateEmptyFileOnPod/" \
    -e "s/framework.DefaultPodDeletionTimeout\b/e2epod.DefaultPodDeletionTimeout/" \
    -e "s/framework.DumpAllNamespaceInfo\b/e2edebug.DumpAllNamespaceInfo/" \
    -e "s/framework.DumpDebugInfo\b/e2eoutput.DumpDebugInfo/" \
    -e "s/framework.DumpNodeDebugInfo\b/e2edebug.DumpNodeDebugInfo/" \
    -e "s/framework.EtcdUpgrade\b/e2eproviders.EtcdUpgrade/" \
    -e "s/framework.EventsLister\b/e2edebug.EventsLister/" \
    -e "s/framework.ExecOptions\b/e2epod.ExecOptions/" \
    -e "s/framework.ExpectNodeHasLabel\b/e2enode.ExpectNodeHasLabel/" \
    -e "s/framework.ExpectNodeHasTaint\b/e2enode.ExpectNodeHasTaint/" \
    -e "s/framework.GCEUpgradeScript\b/e2eproviders.GCEUpgradeScript/" \
    -e "s/framework.ImagePrePullList\b/e2epod.ImagePrePullList/" \
    -e "s/framework.KubectlBuilder\b/e2ekubectl.KubectlBuilder/" \
    -e "s/framework.LocationParamGKE\b/e2eproviders.LocationParamGKE/" \
    -e "s/framework.LogSizeDataTimeseries\b/e2edebug.LogSizeDataTimeseries/" \
    -e "s/framework.LogSizeGatherer\b/e2edebug.LogSizeGatherer/" \
    -e "s/framework.LogsSizeData\b/e2edebug.LogsSizeData/" \
    -e "s/framework.LogsSizeDataSummary\b/e2edebug.LogsSizeDataSummary/" \
    -e "s/framework.LogsSizeVerifier\b/e2edebug.LogsSizeVerifier/" \
    -e "s/framework.LookForStringInLog\b/e2eoutput.LookForStringInLog/" \
    -e "s/framework.LookForStringInPodExec\b/e2eoutput.LookForStringInPodExec/" \
    -e "s/framework.LookForStringInPodExecToContainer\b/e2eoutput.LookForStringInPodExecToContainer/" \
    -e "s/framework.MasterAndDNSNodes\b/e2edebug.MasterAndDNSNodes/" \
    -e "s/framework.MasterNodes\b/e2edebug.MasterNodes/" \
    -e "s/framework.MasterUpgradeGKE\b/e2eproviders.MasterUpgradeGKE/" \
    -e "s/framework.NewKubectlCommand\b/e2ekubectl.NewKubectlCommand/" \
    -e "s/framework.NewLogsVerifier\b/e2edebug.NewLogsVerifier/" \
    -e "s/framework.NewNodeKiller\b/e2enode.NewNodeKiller/" \
    -e "s/framework.NewResourceUsageGatherer\b/e2edebug.NewResourceUsageGatherer/" \
    -e "s/framework.NodeHasTaint\b/e2enode.NodeHasTaint/" \
    -e "s/framework.NodeKiller\b/e2enode.NodeKiller/" \
    -e "s/framework.NodesSet\b/e2edebug.NodesSet/" \
    -e "s/framework.PodClient\b/e2epod.PodClient/" \
    -e "s/framework.RemoveLabelOffNode\b/e2enode.RemoveLabelOffNode/" \
    -e "s/framework.ResourceConstraint\b/e2edebug.ResourceConstraint/" \
    -e "s/framework.ResourceGathererOptions\b/e2edebug.ResourceGathererOptions/" \
    -e "s/framework.ResourceUsagePerContainer\b/e2edebug.ResourceUsagePerContainer/" \
    -e "s/framework.ResourceUsageSummary\b/e2edebug.ResourceUsageSummary/" \
    -e "s/framework.RunHostCmd\b/e2eoutput.RunHostCmd/" \
    -e "s/framework.RunHostCmdOrDie\b/e2eoutput.RunHostCmdOrDie/" \
    -e "s/framework.RunHostCmdWithFullOutput\b/e2eoutput.RunHostCmdWithFullOutput/" \
    -e "s/framework.RunHostCmdWithRetries\b/e2eoutput.RunHostCmdWithRetries/" \
    -e "s/framework.RunKubectl\b/e2ekubectl.RunKubectl/" \
    -e "s/framework.RunKubectlInput\b/e2ekubectl.RunKubectlInput/" \
    -e "s/framework.RunKubectlOrDie\b/e2ekubectl.RunKubectlOrDie/" \
    -e "s/framework.RunKubectlOrDieInput\b/e2ekubectl.RunKubectlOrDieInput/" \
    -e "s/framework.RunKubectlWithFullOutput\b/e2ekubectl.RunKubectlWithFullOutput/" \
    -e "s/framework.RunKubemciCmd\b/e2ekubectl.RunKubemciCmd/" \
    -e "s/framework.RunKubemciWithKubeconfig\b/e2ekubectl.RunKubemciWithKubeconfig/" \
    -e "s/framework.SingleContainerSummary\b/e2edebug.SingleContainerSummary/" \
    -e "s/framework.SingleLogSummary\b/e2edebug.SingleLogSummary/" \
    -e "s/framework.TimestampedSize\b/e2edebug.TimestampedSize/" \
    -e "s/framework.WaitForAllNodesSchedulable\b/e2enode.WaitForAllNodesSchedulable/" \
    -e "s/framework.WaitForSSHTunnels\b/e2enode.WaitForSSHTunnels/" \
    -e "s/framework.WorkItem\b/e2edebug.WorkItem/" \
    "$@"

for i in "$@"; do
    # Import all sub packages and let goimports figure out which of those
    # are redundant (= already imported) or not needed.
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2edebug "k8s.io/kubernetes/test/e2e/framework/debug"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2ekubectl "k8s.io/kubernetes/test/e2e/framework/kubectl"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2enode "k8s.io/kubernetes/test/e2e/framework/node"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eoutput "k8s.io/kubernetes/test/e2e/framework/pod/output"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2epod "k8s.io/kubernetes/test/e2e/framework/pod"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eproviders "k8s.io/kubernetes/test/e2e/framework/providers"' "$i"
    goimports -w "$i"
done
2022-10-06 08:19:47 +02:00
Patrick Ohly
5614a9d064 e2e framework: eliminate interim sub packages
The "todo" packages were necessary while moving code around to avoid hitting
cyclic dependencies. Now that any sub package can depend on the framework, they
are no longer needed and the code can be moved into the normal sub packages.
2022-10-06 08:16:47 +02:00
Patrick Ohly
a46fea53e6 e2e framework: move pod exec and create into sub package 2022-10-06 08:16:47 +02:00
Humble Chirammal
9e9fc2be88 various corrections in test/e2e package
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 18:59:30 +05:30
Patrick Ohly
24230b4307 e2e storage: support creating ReplicaSet
This is useful for running a driver on a subset of all ready nodes:

- use e2enode.GetBoundedReadySchedulableNodes with a suitable
  maximum number of nodes to determine how much nodes are available
  for a test
- define pod anti-affinity in the PodTemplate:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/instance: xxxxxxx
            topologyKey: kubernetes.io/hostname
- set the ReplicaSetSpec.Replicas value to the number of nodes
2022-09-12 14:38:28 +02:00
Joe Betz
7be77f9ecc changes code to improve output for files under test/e2e/storage 2022-09-02 17:43:41 +05:30
Fabio Bertinatto
e13c9be88a e2e: use custom timeouts in GetSnapshotContentFromSnapshot() 2022-08-26 15:16:35 -03:00
Davanum Srinivas
a9593d634c Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-07-26 13:14:05 -04:00
Abirdcfly
00b9ead02c cleanup: remove duplicate import
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-14 11:25:19 +08:00
Dave Chen
fd4b5b629b Stop using the deprecated method CurrentGinkgoTestDescription
Besides, the using of method might lead to a `concurrent map writes`
issue per the discussion here: https://github.com/onsi/ginkgo/issues/970

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
ece0bb3522 Adapt to new type of GinkgoWriter
Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:53 +08:00
Dave Chen
857458cfa5 update ginkgo from v1 to v2 and gomega to 1.19.0
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:46 +08:00
Kubernetes Prow Robot
313d9fb0b1 Merge pull request #109311 from my-git9/creatego
Modify function parameters
2022-05-05 13:50:41 -07:00
Jordan Liggitt
410ac59c0d Remove PodSecurityPolicy admission plugin 2022-05-04 16:00:56 -04:00
xin.li
e0496cee62 Modify function parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2022-04-05 22:05:39 +08:00
Masaki Kimura
cd634b59ac Implement initial E2E test for AnyVolumeDataSource 2022-04-04 16:10:39 +00:00
Patrick Ohly
ff5558edc3 test: support storage tests with non-standard kubelet root directory
Some storage tests deploy DaemonSets which hard-code /var/lib/kubelet as root
directory for kubelet registration and pod directory. There was already a
parameter which allowed specifying the root directory, just with a very
confusing name ("--volume-dir") and matching field name. A --kubelet-root-dir
parameters gets added because this may make it easier to find the parameter,
with the old name preserved as an alias for the same field for backwards
compatibility.
2022-03-03 12:34:18 +01:00
Kubernetes Prow Robot
1f8b1b84fb Merge pull request #102886 from gnufied/add-local-expansion
Add support for expanding local volumes
2021-08-12 11:55:50 -07:00
Hemant Kumar
d166cabd9e Add e2e for local volume expansion 2021-08-06 15:08:33 -04:00
lzj
7cf30d5690 fix/remove-errors-pkg 2021-06-24 16:09:59 +08:00
Patrick Ohly
53af002742 storage e2e: capture driver pod changes, including all events
As seen in https://github.com/kubernetes/kubernetes/issues/102452, we
currently don't have pod events for the CSI driver pods because of the
different namespace and would need them to determine whether the
driver gets evicted.

Previously, only changes of the pods where logged. Perhaps even more
interesting are events in the namespace.
2021-06-02 17:37:03 +02:00
Masaki Kimura
382f37142c storage e2e: auto detect sector size 2021-05-11 22:32:05 +00:00
drfish
244d7a5d67 Remove suffixes for VolumeSnapshotClasses in E2E tests 2021-03-23 21:24:28 +08:00
Benjamin Elder
56e092e382 hack/update-bazel.sh 2021-02-28 15:17:29 -08:00
Jiawei Wang
c4dfee6262 Fix storage e2e snapshot test deletion order 2021-02-17 16:17:08 -08:00
Patrick Ohly
d27c541b45 test: bump CSI container versions for mock testing
These are the latest stable releases. We should test with those.
The newer external-provisioner no longer needs (and doesn't support)
the --provisioner parameter.
2021-02-05 21:08:39 +01:00
Jiawei Wang
356bea6c9f Add storage framework and address comments 2020-12-10 22:48:06 -08:00
Jiawei Wang
988563f8f5 Extract testsuite api to a separate package
Extract TestSuite, TestDriver, TestPattern, TestConfig
and VolumeResource, SnapshotVolumeResource from testsuite
package and put them into a new package called api.

The ultimate goal here is to make the testsuites as clean
as possible. And only testsuites in the package.
2020-12-10 11:12:51 -08:00
Fabio Bertinatto
ee082985c2 e2e: promote use of functions that support custom timeouts in storage tests.
WaitForPodSuccessInNamespace[Slow] are replaced by WaitForPodSuccessInNamespaceTimeout(),
so that custom timeouts are used instead of the hardcoded ones.
2020-12-02 16:14:13 -03:00
Fabio Bertinatto
c82626f96f e2e: use custom timeouts in all storage E2E tests 2020-12-02 15:57:58 -03:00
Christian Huffman
38071e74cf Correct rebase issues 2020-11-12 17:09:49 -05:00
Christian Huffman
4d2d063635 Included e2e test for CSIDriver FSGroupPolicy 2020-11-12 16:30:38 -05:00
Kubernetes Prow Robot
0b1d94a849 Merge pull request #96479 from jingxu97/nov/testwindows
Update storage test suits for Windows
2020-11-12 13:17:29 -08:00