1
0
mirror of https://github.com/rancher/steve.git synced 2025-07-15 23:46:34 +00:00
Commit Graph

17 Commits

Author SHA1 Message Date
Eric Promislow
d794bfe4e8
Index arbitrary labels (#317)
* Add more fields to index when sql-caching is on.

* Restore the gvkKey helper, add event fields.

The UI team wasn't sure whether the event fields should go in the empty-string group or in 'events.k8s.io',
so let's go with both until/unless specified otherwise.

* More fixes to the fields to index:

- Remove the erroneously added management.cattle.io.nodes fields
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)

* Start on the virtual-field work.

* Map `Event.type` to `Event._type` for indexing.

* Add a unit test for field replacement for Event.type

* Add label processing.

* Don't test for transformation of event objects in the common module.

* Parse metadata.label queries differently.

* Improve a variable name that turned out to not be temporary.

* No need to specifically cache certain labels, as all are now cached.

* Add a test to verify simple label (m.labels.foo=blah) queries work.

* 'addLabelFields' never returns an error.

* Delete superseded function.

* Was calling 'addLabelFields' one time too many.

* Start using k8s ParseToRequirements

* Pull in the k8s parser.

* Successfully test for quotation marks.

* Add quoted strings to the lexer.

* Move to a forked k8s label lexer to include non-label tests.

* Improve and test the way quoted strings in the query are detected.

* Reinstate the original Apache license in the derived code.

Following clause 4.3 of the Apache license:
"You must cause any modified files to carry prominent notices stating that You changed the files..."

* Ignore case for operators.

* Test IN multiple-target-values

* Test the not-in operator.

* Ignore case for operators.

SQL is case-insensitive on field names and values, so this just adds consistency.

* Added tests for parsing EXISTS and NOT-EXISTS queries.

* Parse less-than and greater-than ops

* Lasso's `CacheFor` now takes a `watchable` argument.

* Support 'gt' and 'lt' as synonyms for '<' and '>'.

I see both types of operators being bandied about -- it's easy to support the aliases.

* typo fix

* Have the filter parser allow exist tests only on labels.

Also reduce the case where there's no namespace function.

* Specify hard-wired fields to index alphabetically.

* Remove unused variable.

* Parser: 'metadata.labels[FIELD]' is valid

* Pull in new gvk fields from main (and keep in alpha order).

* Fixed a couple of drops done during the last rebase.

* Add a reminder to keep the entries in alpha order.

* Test TransformLabels

* Remove TransformLabels

* Remove unused/unneeded code.

* Describe diffs between our label-selector parser and upstream's.

* Use the merged lasso 46333 work.

* Drop unused field.

* Tighten up the code.

* Specify which commit the label selector parser is based on.

* Allow both single-quoted and double-quoted value matching, doc difference.

* More review-driven changes:

- Stricter processing of m.l.name keys:
  Require ending close-bracket for a start-bracket

- Comment fix

- Moving sql processing from lasso to steve: some changes missed in rebase

* Drop support for double-quotes for string values.

For now on only single-quotes (or none where possible) are allowed.

* Renaming and dropping an init block.

* Quoted strings are dropped from the filter queries

In particular, label values have a specific syntax: they must
start and end with a letter, and their innards may contain
only alnums '.', '-' and '_'.  So there's no need for quoting.

And that means now that `=` and `==` do exact matches,
and the `~` operator does a partial match.

`!=` and `!~` negate -- note that `!~` is a stricter operation than `!=`,
in that given a set of possible string values, `!=` will accept more
of them than `!~`. Maybe I shouldn't have gone here, but these operators
reminded me of learning about `nicht durfen` and `nicht sollen` in German,
or something like that.

* Move a constant definition to the module level.

* Remove commented-out code.

* Remove unused func and adjacent redundant comment.
2025-01-30 11:57:23 -08:00
Eric Promislow
c1805696ce
SQLite backed cache: Support sorting mgmt clusters on value in a specific condition (#447)
* Replace primary/secondary sort fields with an array of sort directives.

* Allow more than 2 sort-params in a search query.

* Add a virtual 'status.ready' field to clusters.

* Rename status.ready -> status.connected

* Set virtual field 'spec.internal' <- spec.displayName == 'local'

* Need to declare all virtual fields to index.

* Ready clusters have condition[type==Ready && status=True]

* Update the README to reflect generalized sorting.

* Bump lasso to get revised sort directives.

* Review-driven changes, mostly comments and drop unneeded code.

* Add unit tests to verify sort-order stringification.

* Ignore empty-string sort components.

* Fix a rebase mishap.

* Drop unneeded commented-out code.

* Clusters have a 'spec.internal' field, no need to synthesize one.

* Added a note on square-brackets for label references.

This should be added to the README for filter queries in the PR for 46333.

* Bump to latest sqlcache-free lasso
2025-01-27 11:55:09 -08:00
Tom Lebreux
9dd9b0f625
Move lasso SQL cache in Steve (#452)
* Copy pkg/cache/sql from lasso to pkg/sqlcache

* Rename import from github.com/rancher/lasso/pkg/cache/sql to github.com/rancher/steve/pkg/sqlcache

* Fix filter.Match -> filter.Matches

* go mod tidy

* Fix lint errors

* Remove lasso SQL cache mentions

* Fix more CI lint errors

* fix goimports

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* fix tests (Match -> Matches)

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Fix Sort order

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
Co-authored-by: Silvio Moioli <silvio@moioli.net>
2025-01-17 09:34:48 -05:00
Tom Lebreux
4477e2c1c4
Add indexed field Event.involvedObject.uid (#439) 2025-01-14 08:33:44 +01:00
Silvio Moioli
e70d03d6ce
sort typeSpecificIndexedFields lexicographically (#437) 2025-01-11 12:26:44 +01:00
Tom Lebreux
8666ce1f54
Update indexed fields (#433)
* Update indexed fields

* Add .status.currentReplicas
2025-01-09 09:35:49 -05:00
Silvio Moioli
5e15f22b0d
SQLite backed cache: indexed fields round #4 (#430)
* Change indexed fields according to https://github.com/rancher/rancher/issues/48256wq

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Fix typo

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-12-19 11:01:09 -05:00
Silvio Moioli
58f2264f16
sql: silence informer errors on unwatchable resources (#284)
* bump lasso to include pull #111

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Make IsListWatchable public to be reused in other packages

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Let lasso know whether a type is watchable upon requesting a cache

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Adapt existing tests

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Add a test to check watchability is detected correctly

Signed-off-by: Silvio Moioli <silvio@moioli.net>

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-12-19 04:53:10 -05:00
Silvio Moioli
65129458a7
cache/sql: add more indexed fields (#400)
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-12-06 15:59:02 -08:00
Eric Promislow
06c2eb50d1
Index more sqlite cache fields (#271)
* Add more fields to index when sql-caching is on.

Misc changes:
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
- Specify full path to the management.cattle.io fields.
- Map `Event.type` to `Event._type` for indexing.

Use a compound transform-func to first check for a "signal",
and then to run all the relevant transformers until either
one fails or the list is exhausted.

- Includes moving the fakeSummaryCache type into a common area.

Use a simpler way of running transforms.

* Inline the function to get the gvk key.

* Create a '--sql-cache' flag to turn on caching for the steve CLI.

* Improve error-handling in object transformer.

* Drop the 'GetTransform' function.

* Inline the code that transforms a payload into a k8s-unstructured object.
2024-10-18 11:06:29 -07:00
Felipe Gehrke
99e479ba0f
Adding Group, Version, Kind in Update proxy_store.go (#270)
* adding gvk completion to update method in proxy_store
2024-09-13 15:19:58 -03:00
Felipe Gehrke
e930ae3691
Added Namespace check to proxy_store.go Create (#272)
* added namespace check to proxy_store create

* added namespaced resources checks and create tests

* Update pkg/stores/proxy/proxy_store.go

* changed error message and added missing name only test

* updated sql/proxy_store

* changed return to use apierror.NewAPIError

---------

Co-authored-by: Felipe C. Gehrke <felipe@localhost.localdomain>
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2024-09-06 17:25:32 -03:00
Michael Bolot
1149920168 Adding virtual fields
Adds logic which adds virtual fields resources. This allows these fields
to be sorted/filtered on when the SQL cache is enabled. Id and
metadata.state.name were added as the first two fields.
2024-08-28 12:17:53 -05:00
Silvio Moioli
0841e03c57
sql: bugfix: return total resource count correctly (#236)
* sql: drop dead code

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* sql: bugfix: return total resource count correctly

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* adapt tests

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* adapt mocks

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* TEMP: remove this when bumping lasso to include https://github.com/rancher/lasso/pull/84

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Use latest lasso instead of fork

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2024-07-05 16:17:16 -04:00
Silvio Moioli
88fd70abbd sql: drop dead code
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-07-05 16:08:56 +02:00
vardhaman22
dae842ea98 updated wrangler from v2 to v3
also updated k8s dependencies to v0.30.1
2024-06-05 22:53:08 +05:30
Silvio Moioli
7a84620e8b
SQLite backed cache (#223)
This uses SQLite-backed informers provided by Lasso with https://github.com/rancher/lasso/pull/65 to implement Steve API (/v1/) functionality.

This new functionality is available behind a feature flag to be specified at Steve startup

See https://confluence.suse.com/pages/viewpage.action?pageId=1359086083 

Co-authored-by: Ricardo Weir <ricardo.weir@suse.com>
Co-authored-by: Michael Bolot <michael.bolot@suse.com>
Co-authored-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-06-05 16:17:12 +02:00