2024-10-18 18:06:29 +00:00
|
|
|
package virtual_test
|
|
|
|
|
|
|
|
import (
|
2025-01-27 19:55:09 +00:00
|
|
|
"fmt"
|
2024-10-18 18:06:29 +00:00
|
|
|
"strings"
|
|
|
|
"testing"
|
|
|
|
|
Index arbitrary labels (#317)
* Add more fields to index when sql-caching is on.
* Restore the gvkKey helper, add event fields.
The UI team wasn't sure whether the event fields should go in the empty-string group or in 'events.k8s.io',
so let's go with both until/unless specified otherwise.
* More fixes to the fields to index:
- Remove the erroneously added management.cattle.io.nodes fields
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
* Start on the virtual-field work.
* Map `Event.type` to `Event._type` for indexing.
* Add a unit test for field replacement for Event.type
* Add label processing.
* Don't test for transformation of event objects in the common module.
* Parse metadata.label queries differently.
* Improve a variable name that turned out to not be temporary.
* No need to specifically cache certain labels, as all are now cached.
* Add a test to verify simple label (m.labels.foo=blah) queries work.
* 'addLabelFields' never returns an error.
* Delete superseded function.
* Was calling 'addLabelFields' one time too many.
* Start using k8s ParseToRequirements
* Pull in the k8s parser.
* Successfully test for quotation marks.
* Add quoted strings to the lexer.
* Move to a forked k8s label lexer to include non-label tests.
* Improve and test the way quoted strings in the query are detected.
* Reinstate the original Apache license in the derived code.
Following clause 4.3 of the Apache license:
"You must cause any modified files to carry prominent notices stating that You changed the files..."
* Ignore case for operators.
* Test IN multiple-target-values
* Test the not-in operator.
* Ignore case for operators.
SQL is case-insensitive on field names and values, so this just adds consistency.
* Added tests for parsing EXISTS and NOT-EXISTS queries.
* Parse less-than and greater-than ops
* Lasso's `CacheFor` now takes a `watchable` argument.
* Support 'gt' and 'lt' as synonyms for '<' and '>'.
I see both types of operators being bandied about -- it's easy to support the aliases.
* typo fix
* Have the filter parser allow exist tests only on labels.
Also reduce the case where there's no namespace function.
* Specify hard-wired fields to index alphabetically.
* Remove unused variable.
* Parser: 'metadata.labels[FIELD]' is valid
* Pull in new gvk fields from main (and keep in alpha order).
* Fixed a couple of drops done during the last rebase.
* Add a reminder to keep the entries in alpha order.
* Test TransformLabels
* Remove TransformLabels
* Remove unused/unneeded code.
* Describe diffs between our label-selector parser and upstream's.
* Use the merged lasso 46333 work.
* Drop unused field.
* Tighten up the code.
* Specify which commit the label selector parser is based on.
* Allow both single-quoted and double-quoted value matching, doc difference.
* More review-driven changes:
- Stricter processing of m.l.name keys:
Require ending close-bracket for a start-bracket
- Comment fix
- Moving sql processing from lasso to steve: some changes missed in rebase
* Drop support for double-quotes for string values.
For now on only single-quotes (or none where possible) are allowed.
* Renaming and dropping an init block.
* Quoted strings are dropped from the filter queries
In particular, label values have a specific syntax: they must
start and end with a letter, and their innards may contain
only alnums '.', '-' and '_'. So there's no need for quoting.
And that means now that `=` and `==` do exact matches,
and the `~` operator does a partial match.
`!=` and `!~` negate -- note that `!~` is a stricter operation than `!=`,
in that given a set of possible string values, `!=` will accept more
of them than `!~`. Maybe I shouldn't have gone here, but these operators
reminded me of learning about `nicht durfen` and `nicht sollen` in German,
or something like that.
* Move a constant definition to the module level.
* Remove commented-out code.
* Remove unused func and adjacent redundant comment.
2025-01-30 19:57:23 +00:00
|
|
|
"github.com/rancher/steve/pkg/resources/virtual"
|
2024-10-18 18:06:29 +00:00
|
|
|
"github.com/rancher/steve/pkg/resources/virtual/common"
|
|
|
|
"github.com/rancher/steve/pkg/summarycache"
|
|
|
|
"github.com/rancher/wrangler/v3/pkg/summary"
|
|
|
|
"github.com/stretchr/testify/require"
|
|
|
|
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
|
|
|
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
|
Index arbitrary labels (#317)
* Add more fields to index when sql-caching is on.
* Restore the gvkKey helper, add event fields.
The UI team wasn't sure whether the event fields should go in the empty-string group or in 'events.k8s.io',
so let's go with both until/unless specified otherwise.
* More fixes to the fields to index:
- Remove the erroneously added management.cattle.io.nodes fields
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
* Start on the virtual-field work.
* Map `Event.type` to `Event._type` for indexing.
* Add a unit test for field replacement for Event.type
* Add label processing.
* Don't test for transformation of event objects in the common module.
* Parse metadata.label queries differently.
* Improve a variable name that turned out to not be temporary.
* No need to specifically cache certain labels, as all are now cached.
* Add a test to verify simple label (m.labels.foo=blah) queries work.
* 'addLabelFields' never returns an error.
* Delete superseded function.
* Was calling 'addLabelFields' one time too many.
* Start using k8s ParseToRequirements
* Pull in the k8s parser.
* Successfully test for quotation marks.
* Add quoted strings to the lexer.
* Move to a forked k8s label lexer to include non-label tests.
* Improve and test the way quoted strings in the query are detected.
* Reinstate the original Apache license in the derived code.
Following clause 4.3 of the Apache license:
"You must cause any modified files to carry prominent notices stating that You changed the files..."
* Ignore case for operators.
* Test IN multiple-target-values
* Test the not-in operator.
* Ignore case for operators.
SQL is case-insensitive on field names and values, so this just adds consistency.
* Added tests for parsing EXISTS and NOT-EXISTS queries.
* Parse less-than and greater-than ops
* Lasso's `CacheFor` now takes a `watchable` argument.
* Support 'gt' and 'lt' as synonyms for '<' and '>'.
I see both types of operators being bandied about -- it's easy to support the aliases.
* typo fix
* Have the filter parser allow exist tests only on labels.
Also reduce the case where there's no namespace function.
* Specify hard-wired fields to index alphabetically.
* Remove unused variable.
* Parser: 'metadata.labels[FIELD]' is valid
* Pull in new gvk fields from main (and keep in alpha order).
* Fixed a couple of drops done during the last rebase.
* Add a reminder to keep the entries in alpha order.
* Test TransformLabels
* Remove TransformLabels
* Remove unused/unneeded code.
* Describe diffs between our label-selector parser and upstream's.
* Use the merged lasso 46333 work.
* Drop unused field.
* Tighten up the code.
* Specify which commit the label selector parser is based on.
* Allow both single-quoted and double-quoted value matching, doc difference.
* More review-driven changes:
- Stricter processing of m.l.name keys:
Require ending close-bracket for a start-bracket
- Comment fix
- Moving sql processing from lasso to steve: some changes missed in rebase
* Drop support for double-quotes for string values.
For now on only single-quotes (or none where possible) are allowed.
* Renaming and dropping an init block.
* Quoted strings are dropped from the filter queries
In particular, label values have a specific syntax: they must
start and end with a letter, and their innards may contain
only alnums '.', '-' and '_'. So there's no need for quoting.
And that means now that `=` and `==` do exact matches,
and the `~` operator does a partial match.
`!=` and `!~` negate -- note that `!~` is a stricter operation than `!=`,
in that given a set of possible string values, `!=` will accept more
of them than `!~`. Maybe I shouldn't have gone here, but these operators
reminded me of learning about `nicht durfen` and `nicht sollen` in German,
or something like that.
* Move a constant definition to the module level.
* Remove commented-out code.
* Remove unused func and adjacent redundant comment.
2025-01-30 19:57:23 +00:00
|
|
|
"k8s.io/apimachinery/pkg/runtime/schema"
|
2024-10-18 18:06:29 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
func TestTransformChain(t *testing.T) {
|
|
|
|
tests := []struct {
|
|
|
|
name string
|
|
|
|
input any
|
|
|
|
hasSummary *summary.SummarizedObject
|
|
|
|
hasRelationships []summarycache.Relationship
|
|
|
|
wantOutput any
|
|
|
|
wantError bool
|
|
|
|
}{
|
|
|
|
{
|
|
|
|
name: "add summary + relationships + reserved fields",
|
|
|
|
hasSummary: &summary.SummarizedObject{
|
|
|
|
PartialObjectMetadata: v1.PartialObjectMetadata{
|
|
|
|
ObjectMeta: v1.ObjectMeta{
|
|
|
|
Name: "testobj",
|
|
|
|
Namespace: "test-ns",
|
|
|
|
},
|
|
|
|
TypeMeta: v1.TypeMeta{
|
|
|
|
APIVersion: "test.cattle.io/v1",
|
|
|
|
Kind: "TestResource",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
Summary: summary.Summary{
|
|
|
|
State: "success",
|
|
|
|
Transitioning: false,
|
|
|
|
Error: false,
|
|
|
|
Message: []string{"resource 1 rolled out", "resource 2 rolled out"},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
hasRelationships: []summarycache.Relationship{
|
|
|
|
{
|
|
|
|
ToID: "1345",
|
|
|
|
ToType: "SomeType",
|
|
|
|
ToNamespace: "some-ns",
|
|
|
|
FromID: "78901",
|
|
|
|
FromType: "TestResource",
|
|
|
|
Rel: "uses",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
input: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "test.cattle.io/v1",
|
|
|
|
"kind": "TestResource",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "testobj",
|
|
|
|
"namespace": "test-ns",
|
|
|
|
},
|
|
|
|
"id": "old-id",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
wantOutput: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "test.cattle.io/v1",
|
|
|
|
"kind": "TestResource",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "testobj",
|
|
|
|
"namespace": "test-ns",
|
|
|
|
"state": map[string]interface{}{
|
|
|
|
"name": "success",
|
|
|
|
"error": false,
|
|
|
|
"transitioning": false,
|
|
|
|
"message": "resource 1 rolled out:resource 2 rolled out",
|
|
|
|
},
|
|
|
|
"relationships": []any{
|
|
|
|
map[string]any{
|
|
|
|
"toId": "1345",
|
|
|
|
"toType": "SomeType",
|
|
|
|
"toNamespace": "some-ns",
|
|
|
|
"fromId": "78901",
|
|
|
|
"fromType": "TestResource",
|
|
|
|
"rel": "uses",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
"id": "test-ns/testobj",
|
|
|
|
"_id": "old-id",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
{
|
|
|
|
name: "processable event",
|
|
|
|
input: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "/v1",
|
|
|
|
"kind": "Event",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "oswaldsFarm",
|
|
|
|
"namespace": "oswaldsNamespace",
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"status": "False",
|
|
|
|
"reason": "Error",
|
|
|
|
"message": "some error",
|
|
|
|
"lastTransitionTime": "2024-01-01",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
"id": "eventTest2id",
|
|
|
|
"type": "Gorniplatz",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
wantOutput: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "/v1",
|
|
|
|
"kind": "Event",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "oswaldsFarm",
|
|
|
|
"namespace": "oswaldsNamespace",
|
|
|
|
"relationships": []any(nil),
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"status": "False",
|
|
|
|
"reason": "Error",
|
|
|
|
"transitioning": false,
|
|
|
|
"error": true,
|
|
|
|
"message": "some error",
|
|
|
|
"lastTransitionTime": "2024-01-01",
|
|
|
|
"lastUpdateTime": "2024-01-01",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
"id": "oswaldsNamespace/oswaldsFarm",
|
|
|
|
"_id": "eventTest2id",
|
|
|
|
"type": "Gorniplatz",
|
|
|
|
"_type": "Gorniplatz",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
{
|
|
|
|
name: "don't fix non-default-group event fields",
|
|
|
|
input: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "palau.io/v1",
|
|
|
|
"kind": "Event",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "gregsFarm",
|
|
|
|
"namespace": "gregsNamespace",
|
|
|
|
"relationships": []any(nil),
|
|
|
|
},
|
|
|
|
"id": "eventTest1id",
|
|
|
|
"type": "Gorniplatz",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
wantOutput: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "palau.io/v1",
|
|
|
|
"kind": "Event",
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "gregsFarm",
|
|
|
|
"namespace": "gregsNamespace",
|
|
|
|
"relationships": []any(nil),
|
|
|
|
},
|
|
|
|
"id": "gregsNamespace/gregsFarm",
|
|
|
|
"_id": "eventTest1id",
|
|
|
|
"type": "Gorniplatz",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
2025-01-27 19:55:09 +00:00
|
|
|
{
|
|
|
|
name: "a non-ready cluster",
|
|
|
|
input: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "management.cattle.io/v3",
|
|
|
|
"kind": "Cluster",
|
|
|
|
"id": 1,
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "c-m-boris",
|
|
|
|
},
|
|
|
|
"spec": map[string]interface{}{
|
|
|
|
"displayName": "boris",
|
|
|
|
"internal": false,
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "BackingNamespaceCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "DefaultProjectCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "SystemProjectCreated",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
wantOutput: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "management.cattle.io/v3",
|
|
|
|
"kind": "Cluster",
|
|
|
|
"id": "c-m-boris",
|
|
|
|
"_id": 1,
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "c-m-boris",
|
|
|
|
"relationships": []any(nil),
|
|
|
|
},
|
|
|
|
"spec": map[string]interface{}{
|
|
|
|
"displayName": "boris",
|
|
|
|
"internal": false,
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "BackingNamespaceCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "DefaultProjectCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "SystemProjectCreated",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
"connected": false,
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
{
|
|
|
|
name: "a ready cluster",
|
|
|
|
input: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "management.cattle.io/v3",
|
|
|
|
"kind": "Cluster",
|
|
|
|
"id": 2,
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "c-m-natasha",
|
|
|
|
},
|
|
|
|
"spec": map[string]interface{}{
|
|
|
|
"displayName": "natasha",
|
|
|
|
"internal": false,
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "BackingNamespaceCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "Ready",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "SystemProjectCreated",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
wantOutput: &unstructured.Unstructured{
|
|
|
|
Object: map[string]interface{}{
|
|
|
|
"apiVersion": "management.cattle.io/v3",
|
|
|
|
"kind": "Cluster",
|
|
|
|
"id": "c-m-natasha",
|
|
|
|
"_id": 2,
|
|
|
|
"metadata": map[string]interface{}{
|
|
|
|
"name": "c-m-natasha",
|
|
|
|
"relationships": []any(nil),
|
|
|
|
},
|
|
|
|
"spec": map[string]interface{}{
|
|
|
|
"displayName": "natasha",
|
|
|
|
"internal": false,
|
|
|
|
},
|
|
|
|
"status": map[string]interface{}{
|
|
|
|
"conditions": []interface{}{
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "BackingNamespaceCreated",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "Ready",
|
|
|
|
},
|
|
|
|
map[string]interface{}{
|
|
|
|
"error": false,
|
|
|
|
"lastUpdateTime": "2025-01-10T22:52:16Z",
|
|
|
|
"status": "True",
|
|
|
|
"transitioning": false,
|
|
|
|
"type": "SystemProjectCreated",
|
|
|
|
},
|
|
|
|
},
|
|
|
|
"connected": true,
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
2024-10-18 18:06:29 +00:00
|
|
|
}
|
|
|
|
for _, test := range tests {
|
|
|
|
t.Run(test.name, func(t *testing.T) {
|
|
|
|
fakeCache := common.FakeSummaryCache{
|
|
|
|
SummarizedObject: test.hasSummary,
|
|
|
|
Relationships: test.hasRelationships,
|
|
|
|
}
|
|
|
|
tb := virtual.NewTransformBuilder(&fakeCache)
|
|
|
|
raw, isSignal, err := common.GetUnstructured(test.input)
|
|
|
|
require.False(t, isSignal)
|
|
|
|
require.Nil(t, err)
|
|
|
|
apiVersion := raw.GetAPIVersion()
|
|
|
|
parts := strings.Split(apiVersion, "/")
|
|
|
|
gvk := schema.GroupVersionKind{Group: parts[0], Version: parts[1], Kind: raw.GetKind()}
|
2025-01-27 19:55:09 +00:00
|
|
|
if test.name == "a non-ready cluster" {
|
|
|
|
fmt.Printf("Stop here")
|
|
|
|
}
|
2024-10-18 18:06:29 +00:00
|
|
|
output, err := tb.GetTransformFunc(gvk)(test.input)
|
|
|
|
require.Equal(t, test.wantOutput, output)
|
|
|
|
if test.wantError {
|
|
|
|
require.Error(t, err)
|
|
|
|
} else {
|
|
|
|
require.NoError(t, err)
|
|
|
|
}
|
|
|
|
})
|
|
|
|
}
|
|
|
|
}
|