1
0
mirror of https://github.com/rancher/steve.git synced 2025-04-27 19:05:09 +00:00
Commit Graph

585 Commits

Author SHA1 Message Date
renovate-rancher[bot]
fbea880f9d
chore(deps): update module golang.org/x/sync to v0.13.0 2025-04-09 04:38:55 +00:00
Sakala Venkata Krishna Rohit
da9bc2a114
Update urfave/cli to v2 (#523) 2025-04-08 12:46:32 -07:00
Sakala Venkata Krishna Rohit
5ae8585e5d
Fix namespace access control in steve (#568)
* Fix adding namespace resource access

* Add tests for addResourceAccess func
2025-04-07 17:40:43 -07:00
Tom Lebreux
e1061a86cd
Update branch (#549)
* Update branch

* Update norman and apiserver to v0.6.0
2025-03-18 11:36:01 -04:00
Tom Lebreux
b29f7d505c
Reapply "Fix not finding ClusterRoleBinding or RoleBinding for service account…" (#540) (#550)
This reverts commit 53f2165aee.
2025-03-14 11:52:28 -04:00
Tom Lebreux
53f2165aee
Revert "Fix not finding ClusterRoleBinding or RoleBinding for service account…" (#540)
This reverts commit 0f4fd1fde6.
2025-03-11 11:14:21 -04:00
Tom Lebreux
0f4fd1fde6
Fix not finding ClusterRoleBinding or RoleBinding for service accounts (#539) 2025-03-10 13:35:01 -04:00
Eric Promislow
f486902100
Add support for another field: (#532)
provisioning.cattle.io.clusters\
metadata.annotations[provisioning.cattle.io/management-cluster-display-name]

Needed to add another character to the subfieldRegex in
listoption_indexer to allow hyphens in annotation field names.
2025-03-05 12:53:59 -08:00
Eric Promislow
2f331b1a1a
Ensure complex accessors from schema don't result in double-bracketing. (#531) 2025-03-04 09:52:35 -08:00
Eric Promislow
2711fd1f46
Added more fields - part 6. (#485)
* Added more fields - part 6.

* Tweaks needed to get 'batch.cronjobs' working

* Add a clarifying comment on pulling values out of an array.

* Use the correct capitalization for the etcd-snapshot kind.
2025-03-04 09:31:21 -08:00
Eric Promislow
0edba0da3e
Sort labels (#527)
* Support sorting on metadata.labels.NAME

The key to doing this is if we want to sort on, say, `metadata.labels.foo`, we need to
search for all rows with a label of the name `foo` in all the various
join tables we create for each label the query references.

We ignore nulls by giving them lowest priority using "NULLS LAST"
("NULLS FIRST" if sorting in descending order).

* Ensure labels that are mentioned only in sort params are still selected.

If we don't do this -- say we sort on metadata.labels.foo but never
make a test on it, the sort resuilts are ignored.

* Remove extraneous debugger statements.
2025-03-04 09:30:14 -08:00
renovate-rancher[bot]
a85993129e
chore(deps): update module modernc.org/sqlite to v1.36.0 (#528)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-27 17:10:58 -05:00
Eric Promislow
3b45729415
Generate field names with brackets when needed. (#477)
* Generate field names with brackets when needed.

* Stop hard-wiring complex selectors as `["field1", "field2[sub-field3]"]`

and instead represent them as a more consistent `["field1", "field2", "sub-field3"]`

* Convert all filter strings ending with square brackets to an array.

Stop special-casing 'metadata.labels[X]' and handle any query string that ends with '[...]'.

* Stop checking for pre-bracketed terms in constant field-accessor arrays.

In this commit we stop converting string arrays like
`["metadata", "labels[k8s.io/deepcode]"]` into the database field
`"metadata.labels[k8s.io/deepcode]"` and instead will do a
naive `join` to give `metadata[labels[k8s.io/deepcode]]`.  The solution
is to never express the above terms in separate fields, like
`["metadata", "labels", "k8s.io/deepcode"]`. This also better reflects
the stucture of the referenced object.

* gofmt changes

* Simplify comment about 'smartJoin'.
2025-02-25 10:39:29 -08:00
Sakala Venkata Krishna Rohit
527d44a3a7
Bump golang lint (#524) 2025-02-24 09:51:38 -08:00
renovate-rancher[bot]
078ddfe47d
chore(deps): update module github.com/rancher/saml to v0.4.14 (#496)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-23 10:59:59 -08:00
Sakala Venkata Krishna Rohit
0c73b199be
Update renovate.json to ignore packages (#516)
* Rancher 2.7 is EOL, so removing release/v0.1

* Ignore dependencies for steve renovate

* Ignore major, minor bumps of rancher deps

* test
2025-02-21 10:37:19 -08:00
Eric Promislow
c906c36bc3
support unwatchables in vai (#458)
* Create and use a synthetic watcher for non-watchable resources.

* Write unit tests for the synthetic watcher.

* Make the refresh interval for synthetic watchers configurable.

The default is to call `client.List(...)` every 5 seconds for each unwatchable GVK.

There are currently only 3 such GVKs right now so this will be informative
enough but not really noticeable.

* Pass the context into the synthetic watch func.

* Restore changes lost in rebasing.

---------

Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2025-02-20 12:45:58 -08:00
Alejandro Ruiz
6559fa9ad7
Fix "Unable to cancel request for *client.addQuery" warnings (#512) 2025-02-20 12:03:03 -05:00
renovate-rancher[bot]
d7746fd993
chore(deps): update module modernc.org/sqlite to v1.35.0 (#454)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-14 09:05:49 -08:00
Chad Roberts
5b5db5c40f
[main&2.10.3] Add schema links and resource methods for resource verb patch (#450)
* Show patch link on the API resource when patch permission is present and add patch ResourceMethod to the schema.

* Added tests for new functionality and corrected disallowed method for patch
2025-02-14 06:12:17 -05:00
renovate-rancher[bot]
f51f89196c
chore(deps): update module golang.org/x/sync to v0.11.0 (#487)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-13 10:07:08 -08:00
Chad Roberts
ccbadbd75f
Update to k8s 1.32 libraries (#491)
* Rebase

* Update dynamiclistener to v0.6.2-rc.3
2025-02-12 13:26:58 -05:00
renovate-rancher[bot]
36b97a97fb
chore(deps): update actions/checkout action to v4.2.2 (#339)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-12 09:26:50 -08:00
Richard Cox
eab91bb3d3
Fix code blocks in README (#490)
- an errant set of backticks broke quotes below
2025-02-12 09:26:25 -08:00
renovate-rancher[bot]
f664022933
chore(deps): update module github.com/rancher/dynamiclistener to v0.6.1 (#334)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-12 09:20:45 -08:00
renovate-rancher[bot]
d314442a42
chore(deps): update module github.com/urfave/cli/v2 to v2.27.5 (#336)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-12 09:20:20 -08:00
renovate-rancher[bot]
b72a24c0bc
chore(deps): update module github.com/google/gnostic-models to v0.6.9 (#372)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-12 09:17:27 -08:00
Silvio Moioli
3350323f91
sql: propagate and use contexts (#465)
Previous SQLite-related code used context.Background() and context.TODO() because it was not developed with context awareness.

This commit propagates the main Steve context so that it can be used when interacting with SQL context-aware functions.

This PR removes all production-code use of context.Background() and context.TODO() and replaces test-code use of TODO with Background.

Contributes to rancher/rancher#47825
2025-02-12 09:46:10 +01:00
renovate-rancher[bot]
e71f8c455d
chore(deps): update module github.com/rancher/wrangler/v3 to v3.1.0 (#344)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 13:04:45 -08:00
renovate-rancher[bot]
b4551c5472
chore(deps): update module go.uber.org/mock to v0.5.0 (#345)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 13:03:15 -08:00
renovate-rancher[bot]
cab3ff10b5
chore(deps): update golangci/golangci-lint-action action to v6.3.2 (#341)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:54:07 -08:00
renovate-rancher[bot]
ac1e70719e
chore(deps): update actions/setup-go action to v5.3.0 (#340)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:53:45 -08:00
renovate-rancher[bot]
747284103a
chore(deps): update module github.com/urfave/cli to v1.22.16 (#335)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:53:19 -08:00
renovate-rancher[bot]
b826f74af2
chore(deps): update module github.com/adrg/xdg to v0.5.3 (#332)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:52:43 -08:00
renovate-rancher[bot]
8c96897309
chore(deps): update dependency go to v1.23.6 (#331)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:52:20 -08:00
renovate-rancher[bot]
4768bba63b
chore(deps): update module github.com/gorilla/websocket to v1.5.3 (#333)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:52:00 -08:00
renovate-rancher[bot]
43f0f7cf62
Add initial Renovate configuration (#457)
Co-authored-by: renovate-rancher[bot] <119870437+renovate-rancher[bot]@users.noreply.github.com>
2025-02-11 12:27:15 -08:00
Silvio Moioli
9139e492e0
vai: do not discard closing statement errors (#462) 2025-02-07 08:31:08 +01:00
Silvio Moioli
772dc7577e
sql: use a closure to wrap transactions (#469)
This introduces the a `WithTransaction` function, which is then used for all transactional work in Steve.

Because `WithTransaction` takes care of all `Begin`s, `Commit`s and `Rollback`s, it eliminates the problem where forgotten open transactions can block all other operations (with long stalling and `SQLITE_BUSY` errors).

This also:

- merges together the disparate `DBClient` interfaces in one only `db.Client` interface with one unexported non-test implementation. I found this much easier to follow
- refactors the transaction package in order to make it as minimal as possible, and as close to the wrapped `sql.Tx` and `sql.Stmt` functions as possible, in order to reduce cognitive load when working with this part of the codebase
- simplifies tests accordingly
- adds a couple of known files to `.gitignore`
    
Credits to @tomleb for suggesting the approach: https://github.com/rancher/lasso/pull/121#pullrequestreview-2515872507
2025-02-05 10:05:52 +01:00
Silvio Moioli
6a46a1e091
sql: encrypt Tokens by default (#467)
Co-authored-by: Eric Promislow <epromislow@suse.com>
2025-02-03 09:12:02 +01:00
Eric Promislow
d794bfe4e8
Index arbitrary labels (#317)
* Add more fields to index when sql-caching is on.

* Restore the gvkKey helper, add event fields.

The UI team wasn't sure whether the event fields should go in the empty-string group or in 'events.k8s.io',
so let's go with both until/unless specified otherwise.

* More fixes to the fields to index:

- Remove the erroneously added management.cattle.io.nodes fields
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)

* Start on the virtual-field work.

* Map `Event.type` to `Event._type` for indexing.

* Add a unit test for field replacement for Event.type

* Add label processing.

* Don't test for transformation of event objects in the common module.

* Parse metadata.label queries differently.

* Improve a variable name that turned out to not be temporary.

* No need to specifically cache certain labels, as all are now cached.

* Add a test to verify simple label (m.labels.foo=blah) queries work.

* 'addLabelFields' never returns an error.

* Delete superseded function.

* Was calling 'addLabelFields' one time too many.

* Start using k8s ParseToRequirements

* Pull in the k8s parser.

* Successfully test for quotation marks.

* Add quoted strings to the lexer.

* Move to a forked k8s label lexer to include non-label tests.

* Improve and test the way quoted strings in the query are detected.

* Reinstate the original Apache license in the derived code.

Following clause 4.3 of the Apache license:
"You must cause any modified files to carry prominent notices stating that You changed the files..."

* Ignore case for operators.

* Test IN multiple-target-values

* Test the not-in operator.

* Ignore case for operators.

SQL is case-insensitive on field names and values, so this just adds consistency.

* Added tests for parsing EXISTS and NOT-EXISTS queries.

* Parse less-than and greater-than ops

* Lasso's `CacheFor` now takes a `watchable` argument.

* Support 'gt' and 'lt' as synonyms for '<' and '>'.

I see both types of operators being bandied about -- it's easy to support the aliases.

* typo fix

* Have the filter parser allow exist tests only on labels.

Also reduce the case where there's no namespace function.

* Specify hard-wired fields to index alphabetically.

* Remove unused variable.

* Parser: 'metadata.labels[FIELD]' is valid

* Pull in new gvk fields from main (and keep in alpha order).

* Fixed a couple of drops done during the last rebase.

* Add a reminder to keep the entries in alpha order.

* Test TransformLabels

* Remove TransformLabels

* Remove unused/unneeded code.

* Describe diffs between our label-selector parser and upstream's.

* Use the merged lasso 46333 work.

* Drop unused field.

* Tighten up the code.

* Specify which commit the label selector parser is based on.

* Allow both single-quoted and double-quoted value matching, doc difference.

* More review-driven changes:

- Stricter processing of m.l.name keys:
  Require ending close-bracket for a start-bracket

- Comment fix

- Moving sql processing from lasso to steve: some changes missed in rebase

* Drop support for double-quotes for string values.

For now on only single-quotes (or none where possible) are allowed.

* Renaming and dropping an init block.

* Quoted strings are dropped from the filter queries

In particular, label values have a specific syntax: they must
start and end with a letter, and their innards may contain
only alnums '.', '-' and '_'.  So there's no need for quoting.

And that means now that `=` and `==` do exact matches,
and the `~` operator does a partial match.

`!=` and `!~` negate -- note that `!~` is a stricter operation than `!=`,
in that given a set of possible string values, `!=` will accept more
of them than `!~`. Maybe I shouldn't have gone here, but these operators
reminded me of learning about `nicht durfen` and `nicht sollen` in German,
or something like that.

* Move a constant definition to the module level.

* Remove commented-out code.

* Remove unused func and adjacent redundant comment.
2025-01-30 11:57:23 -08:00
Vatsal Parekh
7da1377c89
[main] Upgrade go version to 1.23 (#446)
* Upgrade go version to 1.23

Signed-off-by: Vatsal Parekh <vatsalparekh@outlook.com>

* Upgrade golangci-lint to 1.63.4

Signed-off-by: Vatsal Parekh <vatsalparekh@outlook.com>

---------

Signed-off-by: Vatsal Parekh <vatsalparekh@outlook.com>
2025-01-29 09:05:04 -05:00
Alejandro Ruiz
8ae74491e7
Fix race in accessstore test (#466) 2025-01-28 20:35:24 -05:00
Josh Meranda
5cdbd29ebe
Imperative api pls (#434)
* Add aggregation layer support

* prefer testing.Cleanup

* add sni certs to server opts

* test cleanup

* append snicerts instead of overwriting

---------

Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
Co-authored-by: joshmeranda <joshua.meranda@gmail.com>
2025-01-28 09:08:20 -05:00
Silvio Moioli
ae4153b712
vai: document resync period (#463) 2025-01-28 09:02:38 +01:00
Eric Promislow
c1805696ce
SQLite backed cache: Support sorting mgmt clusters on value in a specific condition (#447)
* Replace primary/secondary sort fields with an array of sort directives.

* Allow more than 2 sort-params in a search query.

* Add a virtual 'status.ready' field to clusters.

* Rename status.ready -> status.connected

* Set virtual field 'spec.internal' <- spec.displayName == 'local'

* Need to declare all virtual fields to index.

* Ready clusters have condition[type==Ready && status=True]

* Update the README to reflect generalized sorting.

* Bump lasso to get revised sort directives.

* Review-driven changes, mostly comments and drop unneeded code.

* Add unit tests to verify sort-order stringification.

* Ignore empty-string sort components.

* Fix a rebase mishap.

* Drop unneeded commented-out code.

* Clusters have a 'spec.internal' field, no need to synthesize one.

* Added a note on square-brackets for label references.

This should be added to the README for filter queries in the PR for 46333.

* Bump to latest sqlcache-free lasso
2025-01-27 11:55:09 -08:00
Tom Lebreux
809e927a0c
Update extension apiserver documentation (#456) 2025-01-21 11:04:13 -05:00
Tom Lebreux
469208d806
Fix sort field parsing (#455) 2025-01-21 10:38:23 +01:00
Tom Lebreux
9dd9b0f625
Move lasso SQL cache in Steve (#452)
* Copy pkg/cache/sql from lasso to pkg/sqlcache

* Rename import from github.com/rancher/lasso/pkg/cache/sql to github.com/rancher/steve/pkg/sqlcache

* Fix filter.Match -> filter.Matches

* go mod tidy

* Fix lint errors

* Remove lasso SQL cache mentions

* Fix more CI lint errors

* fix goimports

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* fix tests (Match -> Matches)

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Fix Sort order

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
Co-authored-by: Silvio Moioli <silvio@moioli.net>
2025-01-17 09:34:48 -05:00
Tom Lebreux
d05a48f9cf
Fix overriding update and remove links with the wrong link (#451) 2025-01-15 11:15:47 -08:00