Deprecate WatchFromStorageWithoutResourceVersion

Around the 1.31 release, we discovered that a change introduced in 1.27 allowead
clients to open WATCH requests directly to etcd. This had detrimental consequences,
enabling abusive clients to bypass caching and overwhelm etcd.
Unlike the API server, etcd lacks protection against such behavior.

To mitigate this, we redirected all WATCH requests to be served from the cache.
The WatchFromStorageWithoutResourceVersion feature gate was retained as an escape hatch.
However, since we have no plans to allow direct WATCH requests to etcd again,
this flag is now obsolete.

Direct WATCH requests to etcd offer no advantage, as they don't provide stronger
consistency guarantees. WATCH operations are inherently inconsistent; unlike LIST
operations, they do not confirm the resource version with a quorum. While Kubernetes
uses the WithRequireLeader option on WATCH requests to prevent maintaining connections
to isolated etcd members, the API server provides the same level of guarantee through
its health checks, which fail if it cannot connect to etcd member.  Therefore,
the WatchFromStorageWithoutResourceVersion feature gate can be deprecated and removed.
This commit is contained in:
Marek Siarkowicz 2025-01-31 11:49:28 +01:00
parent 3bc8f01c74
commit 065bf2004d
4 changed files with 7 additions and 21 deletions

View File

@ -368,6 +368,7 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
genericfeatures.WatchFromStorageWithoutResourceVersion: {
{Version: version.MustParse("1.27"), Default: false, PreRelease: featuregate.Beta},
{Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Deprecated, LockToDefault: true},
},
genericfeatures.WatchList: {

View File

@ -407,6 +407,7 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
WatchFromStorageWithoutResourceVersion: {
{Version: version.MustParse("1.27"), Default: false, PreRelease: featuregate.Beta},
{Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Deprecated, LockToDefault: true},
},
WatchList: {

View File

@ -594,27 +594,7 @@ func TestWatchCacheBypass(t *testing.T) {
Predicate: storage.Everything,
})
if err != nil {
t.Errorf("Watch with RV=0 should be served from cache: %v", err)
}
featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.WatchFromStorageWithoutResourceVersion, false)
_, err = proxy.Watch(context.TODO(), "pod/ns", storage.ListOptions{
ResourceVersion: "",
Predicate: storage.Everything,
})
if err != nil {
t.Errorf("With WatchFromStorageWithoutResourceVersion disabled, watch with unset RV should be served from cache: %v", err)
}
// Inject error to underlying layer and check if cacher is not bypassed.
backingStorage.injectError(errDummy)
featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.WatchFromStorageWithoutResourceVersion, true)
_, err = proxy.Watch(context.TODO(), "pod/ns", storage.ListOptions{
ResourceVersion: "",
Predicate: storage.Everything,
})
if !errors.Is(err, errDummy) {
t.Errorf("With WatchFromStorageWithoutResourceVersion enabled, watch with unset RV should be served from storage: %v", err)
t.Errorf("Watch without RV=0 should be served from cache: %v", err)
}
}

View File

@ -1478,6 +1478,10 @@
lockToDefault: false
preRelease: Beta
version: "1.27"
- default: false
lockToDefault: true
preRelease: Deprecated
version: "1.33"
- name: WatchList
versionedSpecs:
- default: false