provisioning.cattle.io.clusters\
metadata.annotations[provisioning.cattle.io/management-cluster-display-name]
Needed to add another character to the subfieldRegex in
listoption_indexer to allow hyphens in annotation field names.
* Added more fields - part 6.
* Tweaks needed to get 'batch.cronjobs' working
* Add a clarifying comment on pulling values out of an array.
* Use the correct capitalization for the etcd-snapshot kind.
* Support sorting on metadata.labels.NAME
The key to doing this is if we want to sort on, say, `metadata.labels.foo`, we need to
search for all rows with a label of the name `foo` in all the various
join tables we create for each label the query references.
We ignore nulls by giving them lowest priority using "NULLS LAST"
("NULLS FIRST" if sorting in descending order).
* Ensure labels that are mentioned only in sort params are still selected.
If we don't do this -- say we sort on metadata.labels.foo but never
make a test on it, the sort resuilts are ignored.
* Remove extraneous debugger statements.
* Generate field names with brackets when needed.
* Stop hard-wiring complex selectors as `["field1", "field2[sub-field3]"]`
and instead represent them as a more consistent `["field1", "field2", "sub-field3"]`
* Convert all filter strings ending with square brackets to an array.
Stop special-casing 'metadata.labels[X]' and handle any query string that ends with '[...]'.
* Stop checking for pre-bracketed terms in constant field-accessor arrays.
In this commit we stop converting string arrays like
`["metadata", "labels[k8s.io/deepcode]"]` into the database field
`"metadata.labels[k8s.io/deepcode]"` and instead will do a
naive `join` to give `metadata[labels[k8s.io/deepcode]]`. The solution
is to never express the above terms in separate fields, like
`["metadata", "labels", "k8s.io/deepcode"]`. This also better reflects
the stucture of the referenced object.
* gofmt changes
* Simplify comment about 'smartJoin'.
* Create and use a synthetic watcher for non-watchable resources.
* Write unit tests for the synthetic watcher.
* Make the refresh interval for synthetic watchers configurable.
The default is to call `client.List(...)` every 5 seconds for each unwatchable GVK.
There are currently only 3 such GVKs right now so this will be informative
enough but not really noticeable.
* Pass the context into the synthetic watch func.
* Restore changes lost in rebasing.
---------
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
* Show patch link on the API resource when patch permission is present and add patch ResourceMethod to the schema.
* Added tests for new functionality and corrected disallowed method for patch
Previous SQLite-related code used context.Background() and context.TODO() because it was not developed with context awareness.
This commit propagates the main Steve context so that it can be used when interacting with SQL context-aware functions.
This PR removes all production-code use of context.Background() and context.TODO() and replaces test-code use of TODO with Background.
Contributes to rancher/rancher#47825
This introduces the a `WithTransaction` function, which is then used for all transactional work in Steve.
Because `WithTransaction` takes care of all `Begin`s, `Commit`s and `Rollback`s, it eliminates the problem where forgotten open transactions can block all other operations (with long stalling and `SQLITE_BUSY` errors).
This also:
- merges together the disparate `DBClient` interfaces in one only `db.Client` interface with one unexported non-test implementation. I found this much easier to follow
- refactors the transaction package in order to make it as minimal as possible, and as close to the wrapped `sql.Tx` and `sql.Stmt` functions as possible, in order to reduce cognitive load when working with this part of the codebase
- simplifies tests accordingly
- adds a couple of known files to `.gitignore`
Credits to @tomleb for suggesting the approach: https://github.com/rancher/lasso/pull/121#pullrequestreview-2515872507
* Add more fields to index when sql-caching is on.
* Restore the gvkKey helper, add event fields.
The UI team wasn't sure whether the event fields should go in the empty-string group or in 'events.k8s.io',
so let's go with both until/unless specified otherwise.
* More fixes to the fields to index:
- Remove the erroneously added management.cattle.io.nodes fields
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
* Start on the virtual-field work.
* Map `Event.type` to `Event._type` for indexing.
* Add a unit test for field replacement for Event.type
* Add label processing.
* Don't test for transformation of event objects in the common module.
* Parse metadata.label queries differently.
* Improve a variable name that turned out to not be temporary.
* No need to specifically cache certain labels, as all are now cached.
* Add a test to verify simple label (m.labels.foo=blah) queries work.
* 'addLabelFields' never returns an error.
* Delete superseded function.
* Was calling 'addLabelFields' one time too many.
* Start using k8s ParseToRequirements
* Pull in the k8s parser.
* Successfully test for quotation marks.
* Add quoted strings to the lexer.
* Move to a forked k8s label lexer to include non-label tests.
* Improve and test the way quoted strings in the query are detected.
* Reinstate the original Apache license in the derived code.
Following clause 4.3 of the Apache license:
"You must cause any modified files to carry prominent notices stating that You changed the files..."
* Ignore case for operators.
* Test IN multiple-target-values
* Test the not-in operator.
* Ignore case for operators.
SQL is case-insensitive on field names and values, so this just adds consistency.
* Added tests for parsing EXISTS and NOT-EXISTS queries.
* Parse less-than and greater-than ops
* Lasso's `CacheFor` now takes a `watchable` argument.
* Support 'gt' and 'lt' as synonyms for '<' and '>'.
I see both types of operators being bandied about -- it's easy to support the aliases.
* typo fix
* Have the filter parser allow exist tests only on labels.
Also reduce the case where there's no namespace function.
* Specify hard-wired fields to index alphabetically.
* Remove unused variable.
* Parser: 'metadata.labels[FIELD]' is valid
* Pull in new gvk fields from main (and keep in alpha order).
* Fixed a couple of drops done during the last rebase.
* Add a reminder to keep the entries in alpha order.
* Test TransformLabels
* Remove TransformLabels
* Remove unused/unneeded code.
* Describe diffs between our label-selector parser and upstream's.
* Use the merged lasso 46333 work.
* Drop unused field.
* Tighten up the code.
* Specify which commit the label selector parser is based on.
* Allow both single-quoted and double-quoted value matching, doc difference.
* More review-driven changes:
- Stricter processing of m.l.name keys:
Require ending close-bracket for a start-bracket
- Comment fix
- Moving sql processing from lasso to steve: some changes missed in rebase
* Drop support for double-quotes for string values.
For now on only single-quotes (or none where possible) are allowed.
* Renaming and dropping an init block.
* Quoted strings are dropped from the filter queries
In particular, label values have a specific syntax: they must
start and end with a letter, and their innards may contain
only alnums '.', '-' and '_'. So there's no need for quoting.
And that means now that `=` and `==` do exact matches,
and the `~` operator does a partial match.
`!=` and `!~` negate -- note that `!~` is a stricter operation than `!=`,
in that given a set of possible string values, `!=` will accept more
of them than `!~`. Maybe I shouldn't have gone here, but these operators
reminded me of learning about `nicht durfen` and `nicht sollen` in German,
or something like that.
* Move a constant definition to the module level.
* Remove commented-out code.
* Remove unused func and adjacent redundant comment.
* Replace primary/secondary sort fields with an array of sort directives.
* Allow more than 2 sort-params in a search query.
* Add a virtual 'status.ready' field to clusters.
* Rename status.ready -> status.connected
* Set virtual field 'spec.internal' <- spec.displayName == 'local'
* Need to declare all virtual fields to index.
* Ready clusters have condition[type==Ready && status=True]
* Update the README to reflect generalized sorting.
* Bump lasso to get revised sort directives.
* Review-driven changes, mostly comments and drop unneeded code.
* Add unit tests to verify sort-order stringification.
* Ignore empty-string sort components.
* Fix a rebase mishap.
* Drop unneeded commented-out code.
* Clusters have a 'spec.internal' field, no need to synthesize one.
* Added a note on square-brackets for label references.
This should be added to the README for filter queries in the PR for 46333.
* Bump to latest sqlcache-free lasso
* Checkpoint
* Add support for custom columns
* Remove old Store and Delegate abstraction
* Fix nits and rewording
* Remove unused mock file
* Update documentation for extension api server
* Remove the need for scheme for ConvertListOptions
* Rename store to utils
* fixup! Remove the need for scheme for ConvertListOptions
* Move watch helper to tests
* Add convertError at a few places
* Ignore misspell on creater
* Fix comments and remove unused params
* Add convertError to missing error returns
* Fix watcher implementation
* Document request.UserFrom and request.NamespaceFrom
The original PR, steve/pull/158 drifted too far from changes
in main, so it's easier to create a new PR:
* Bring in and update DefaultSchemaTemplatesForStore
* Move from based on `master` to `main`
* Update k8s version
* bump lasso to include pull #111
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Make IsListWatchable public to be reused in other packages
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Let lasso know whether a type is watchable upon requesting a cache
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Adapt existing tests
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Add a test to check watchability is detected correctly
Signed-off-by: Silvio Moioli <silvio@moioli.net>
---------
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* adding NonResourceURLs support to access_store
* added tests to AccessSet NonResourceURLs handling
* change on test script suggested by @tomleb + go mod tidy
* added nonresource to ext api authorization
* added NonResourceURLs implementation in Authorizes + test
* removed non-resource-url tests from the main test
* added new tests for non-resource-urls
* removed unused test data
* changed nonResourceKey to point to struct{}
* addressed comments from @tomleb
* addressed more comments
* fixing typo
* check for empty accessSet
* Refactor ID based partitioning, add unit tests
This resolves an issue where the requested namespace filter was not
always honored.
* Correct naming issues to appease the linter
* changing secret and configmap formatters to return decoded helm data if includeHelmData query parameter is present
* adding tests for gzip magic header
* refactor(accesscontrol): make addAccess directly accept PolicyRules
* refactor(accesscontrol): add new types for encapsulating all needed data
* refactor(accesscontrol): make getRules return resource version
* refactor(accesscontrol): add new getRoleRefs to policyRuleIndex
* refactor(accesscontrol): make accessStore use the new types and method
* cleanup(accesscontrol): remove unused code
* cleanup(accesscontrol): adapt tests
* cleanup(accesscontrol): add some comments and remove unused function
* refactor(accesscontrol): rework indexer to make it more readable and testable
* Fix typo
* test: consistent use of t.Error
* test: refactor policyRulesMock to just use a map
* misc: rename toUserInfo function
* refactor: consistent sort by UID
* Add more fields to index when sql-caching is on.
Misc changes:
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
- Specify full path to the management.cattle.io fields.
- Map `Event.type` to `Event._type` for indexing.
Use a compound transform-func to first check for a "signal",
and then to run all the relevant transformers until either
one fails or the list is exhausted.
- Includes moving the fakeSummaryCache type into a common area.
Use a simpler way of running transforms.
* Inline the function to get the gvk key.
* Create a '--sql-cache' flag to turn on caching for the steve CLI.
* Improve error-handling in object transformer.
* Drop the 'GetTransform' function.
* Inline the code that transforms a payload into a k8s-unstructured object.
Previously, the formatter for state/relationships was disabled when the
sql cache was enabled, since a transform function was adding those
values before they were added to the cache. However, the get/watch calls
currently don't use the cache, causing the state/relationships to be
missing.
This implements the Imperative API that is served at /ext with Steve. The imperative API is compatible with Kubernetes' API server and will be used as an extension API server.
* refactor(accesscontrol): use interface for AccessStore cache
* refactor(accesscontrol): early return when cache is disabled
* test(accesscontrol): add failing unit test
* test(accesscontrol): skip failing test
* added namespace check to proxy_store create
* added namespaced resources checks and create tests
* Update pkg/stores/proxy/proxy_store.go
* changed error message and added missing name only test
* updated sql/proxy_store
* changed return to use apierror.NewAPIError
---------
Co-authored-by: Felipe C. Gehrke <felipe@localhost.localdomain>
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
