1
0
mirror of https://github.com/rancher/steve.git synced 2025-08-28 02:51:18 +00:00
Commit Graph

287 Commits

Author SHA1 Message Date
Silvio Moioli
c48ac64c4d
[v2.10] SQL cache backports (#435)
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2025-01-11 12:26:12 +01:00
nflynt
2175e090fe
Refactor ID based partitioning, add unit tests (#309)
* Refactor ID based partitioning, add unit tests

This resolves an issue where the requested namespace filter was not
always honored.

* Correct naming issues to appease the linter
2024-10-29 09:27:12 -04:00
Diogo Souza
b2f2bab3c4
Helm data formatter (#305)
* changing secret and configmap formatters to return decoded helm data if includeHelmData query parameter is present

* adding tests for gzip magic header
2024-10-28 15:55:36 -05:00
Alejandro Ruiz
fd9a516ecb
refactor(accesscontrol): deterministic cache key hashing (#292)
* refactor(accesscontrol): make addAccess directly accept PolicyRules

* refactor(accesscontrol): add new types for encapsulating all needed data

* refactor(accesscontrol): make getRules return resource version

* refactor(accesscontrol): add new getRoleRefs to policyRuleIndex

* refactor(accesscontrol): make accessStore use the new types and method

* cleanup(accesscontrol): remove unused code

* cleanup(accesscontrol): adapt tests

* cleanup(accesscontrol): add some comments and remove unused function

* refactor(accesscontrol): rework indexer to make it more readable and testable

* Fix typo

* test: consistent use of t.Error

* test: refactor policyRulesMock to just use a map

* misc: rename toUserInfo function

* refactor: consistent sort by UID
2024-10-28 09:35:59 +01:00
Eric Promislow
06c2eb50d1
Index more sqlite cache fields (#271)
* Add more fields to index when sql-caching is on.

Misc changes:
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
- Specify full path to the management.cattle.io fields.
- Map `Event.type` to `Event._type` for indexing.

Use a compound transform-func to first check for a "signal",
and then to run all the relevant transformers until either
one fails or the list is exhausted.

- Includes moving the fakeSummaryCache type into a common area.

Use a simpler way of running transforms.

* Inline the function to get the gvk key.

* Create a '--sql-cache' flag to turn on caching for the steve CLI.

* Improve error-handling in object transformer.

* Drop the 'GetTransform' function.

* Inline the code that transforms a payload into a k8s-unstructured object.
2024-10-18 11:06:29 -07:00
Michael Bolot
f6c6ca839c
Re-adding formatter when SQL cache is enabled (#300)
Previously, the formatter for state/relationships was disabled when the
sql cache was enabled, since a transform function was adding those
values before they were added to the cache. However, the get/watch calls
currently don't use the cache, causing the state/relationships to be
missing.
2024-10-18 12:15:42 -05:00
Tom Lebreux
8fc2dd4f74
Fix ExtensionAPIServer interface (#298)
* Fix comments
* Fix ExtensionAPIServer interface
2024-10-15 10:00:27 -04:00
Tom Lebreux
1f21e5e515
Implement /ext in Steve for Imperative API (#287)
This implements the Imperative API that is served at /ext with Steve. The imperative API is compatible with Kubernetes' API server and will be used as an extension API server.
2024-10-11 15:19:27 -04:00
Eric Promislow
6a11ffb660
Migrate the mocking library to uber's (#291)
* Migrate from golang/mock to uber/mock.

* Update go.mod

* Continue migration to uber/mock.

* Tweaks from running the go.uber.mock's mockgen.
2024-10-10 10:28:25 -07:00
Alejandro Ruiz
484ce1c9e9
refactor(accesscontrol): use interface for AccessStore cache (#290) 2024-10-09 10:21:29 +02:00
Alejandro Ruiz
5c1a56204d
Add tests for concurrent AccessControl store usage (#286)
* refactor(accesscontrol): use interface for AccessStore cache

* refactor(accesscontrol): early return when cache is disabled

* test(accesscontrol): add failing unit test

* test(accesscontrol): skip failing test
2024-10-08 11:18:44 -04:00
Felipe Gehrke
99e479ba0f
Adding Group, Version, Kind in Update proxy_store.go (#270)
* adding gvk completion to update method in proxy_store
2024-09-13 15:19:58 -03:00
Felipe Gehrke
e930ae3691
Added Namespace check to proxy_store.go Create (#272)
* added namespace check to proxy_store create

* added namespaced resources checks and create tests

* Update pkg/stores/proxy/proxy_store.go

* changed error message and added missing name only test

* updated sql/proxy_store

* changed return to use apierror.NewAPIError

---------

Co-authored-by: Felipe C. Gehrke <felipe@localhost.localdomain>
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2024-09-06 17:25:32 -03:00
Michael Bolot
1149920168 Adding virtual fields
Adds logic which adds virtual fields resources. This allows these fields
to be sorted/filtered on when the SQL cache is enabled. Id and
metadata.state.name were added as the first two fields.
2024-08-28 12:17:53 -05:00
Alejandro Ruiz
32c30149a6
refactor(accesscontrol): add unit tests for CacheKey (#245)
This is a follow up to 435e220 with a small refactor to make the function easier to test, and adding unit tests for CacheKey.
2024-08-28 10:06:21 +02:00
Michael Bolot
dfae21cb39 Adding changes from code review 2024-08-05 12:34:59 -05:00
Eric Promislow
8ce0b83be7 Add unit tests for schemaless CRDs 2024-08-05 12:04:00 -05:00
Eric Promislow
72384a606d Handle schema-less CRDs 2024-08-05 12:03:53 -05:00
Alejandro Ruiz
435e220795 fix(accesscontrol): CacheKey could not be stable 2024-07-31 10:24:23 +02:00
Silvio Moioli
0841e03c57
sql: bugfix: return total resource count correctly (#236)
* sql: drop dead code

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* sql: bugfix: return total resource count correctly

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* adapt tests

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* adapt mocks

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* TEMP: remove this when bumping lasso to include https://github.com/rancher/lasso/pull/84

Signed-off-by: Silvio Moioli <silvio@moioli.net>

* Use latest lasso instead of fork

---------

Signed-off-by: Silvio Moioli <silvio@moioli.net>
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
2024-07-05 16:17:16 -04:00
Silvio Moioli
88fd70abbd sql: drop dead code
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-07-05 16:08:56 +02:00
Silvio Moioli
cb311a8ce5 sql: remove spurious log line
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-07-05 10:19:07 +02:00
Michael Bolot
e7a76d14f0 Adding APIGroups back to the schema
Prior schema calculations started with openapiv2 models which included a
model for APIGroups. However, new schema calculations use
ServerGroupsAndResources first, which omitted these values. This
re-adds this type using a static schema.
2024-07-02 13:27:04 -05:00
Tom Lebreux
9ac9be9c0e
Fix missing fields in schema definition endpoint (#215) 2024-06-17 10:52:19 -04:00
vardhaman22
8f3d2e8f21 go generate and go mod tidy 2024-06-05 23:08:15 +05:30
vardhaman22
dae842ea98 updated wrangler from v2 to v3
also updated k8s dependencies to v0.30.1
2024-06-05 22:53:08 +05:30
Silvio Moioli
7a84620e8b
SQLite backed cache (#223)
This uses SQLite-backed informers provided by Lasso with https://github.com/rancher/lasso/pull/65 to implement Steve API (/v1/) functionality.

This new functionality is available behind a feature flag to be specified at Steve startup

See https://confluence.suse.com/pages/viewpage.action?pageId=1359086083 

Co-authored-by: Ricardo Weir <ricardo.weir@suse.com>
Co-authored-by: Michael Bolot <michael.bolot@suse.com>
Co-authored-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-06-05 16:17:12 +02:00
Michael Bolot
c6b887c1cb Fixing schema definitions bugs
Fixes two bugs with the schema definitions:
- Adds resources that are a part of the baseSchema (but aren't k8s resources).
- Adds logic to handle resources which aren't in a prefered version, but
  are still in some version.
2024-05-15 08:43:35 -05:00
Ricardo Weir
d42a5bc545
Skip key generation if cache disabled (#170) 2024-04-19 09:43:15 -04:00
Michael Bolot
bcba9d5e3b
Merge pull request #180 from MbolotSuse/flaky-test
Attempting to fix flaky tests
2024-04-04 08:23:08 -05:00
Michael Bolot
0e9fde750f Attempting to fix flaky tests
Some tests which relied on timeouts were a bit flaky in CI. This PR
refactors a few of them to work on a more reliable method of receiving
from a channel and raises the timeout of another test.
2024-04-03 15:32:23 -05:00
Diogo Souza
46e3638cd8
changing permission for kubeconfig file inside the shell pod (#166)
* changing permission for kubeconfig file inside the shell pod

* creating unit tests for the changes done
2024-04-03 13:33:37 -04:00
Michael Bolot
870824dc8f
Merge pull request #161 from MbolotSuse/resource-schema-improved-cache
Resource schema improved cache
2024-03-14 09:57:06 -05:00
Jonathan Crowther
ca29f475f0
Merge pull request #162 from JonCrowther/count-race-condition
Fix race condition in debounceCounts
2024-03-14 10:52:28 -04:00
Jonathan Crowther
abbe805737 Change which object needs DeepCopy 2024-03-13 19:41:21 -04:00
Michael Bolot
b7618463e6 Fixing bug with preferred versions
In the original implementation of the definition handler, the resource
list was checked for preferred version, and this overrode the preferred
version of the overall group. However, this logic was inaccurate and
did not use the group as the source of truth on the preferred version
like it should have.
2024-03-12 13:59:17 -05:00
Michael Bolot
0f32ff22e0 Adding tests for improved schema cache 2024-03-12 13:53:51 -05:00
Michael Bolot
2f8e64840b Improving Schema Definitions cache/refresh
Updates the schema definitions refresh method to use a new debounce
method. Adds a handler which refreshes the definitions every 2
seconds after adding a CRD and every 10 minutes by default.
2024-03-12 13:49:39 -05:00
Jonathan Crowther
cec59c5a61 Implement custom DeepCopy for Count 2024-03-01 11:56:18 -05:00
Jonathan Crowther
0e74495395 DeepCopy the count before sending it as APIEvent 2024-03-01 10:50:35 -05:00
Michael Bolot
7913f2782a
Merge pull request #141 from moio/watchnames_propagate_errors
WatchNames: return errors via WebSocket
2024-03-01 09:48:41 -06:00
Michael Bolot
099ebc509f Adding tests for schemaDefinitions 2024-02-26 15:42:07 -06:00
Michael Bolot
40f6b10fc7 Adding schema definitions endpoint
Introduces new schema definitions endpoint, allowing the caller to get
the fields/types/descriptions for a given kubernetes resource.
2024-02-26 15:42:03 -06:00
Michael Bolot
3f65942a6a Tests for Reducing number of generated schemas 2024-02-26 15:39:41 -06:00
Michael Bolot
605e74c97a Reducing number of generated schemas
Changes steve to only return top level schema definitions rather
than a full defined schema for every field. Also adds basic docs
and fixes a bug with schema generation on versioned crds
2024-02-26 09:20:42 -06:00
Kevin Joiner
e35a044d5a Bumps rancher/wrangler to it's tagged v2 version.
The previous wrangler commit included all of the v2 changes.
Except for the import paths changes.
2024-01-24 10:24:01 -05:00
Silvio Moioli
d138622063
adapt tests
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2024-01-23 11:25:29 +01:00
Silvio Moioli
1585ed3f7a
WatchNames: return errors via WebSocket
Align behavior with plain Watch, which also does the same.

Fixes https://github.com/rancher/rancher/issues/41809

Signed-off-by: Silvio Moioli <silvio@moioli.net>
2023-12-22 20:10:51 +01:00
Michael Bolot
0b23400f9f Updating ByNames to not return nil, nil
ByNames could previously return a nil value and a nil error. This caused
issues when other parts of the application
(pkg/stores/partition/parallel.go) tried to use the result. Now this
will return an empty list on the error condition, instead of nil
2023-10-16 14:33:35 -05:00
Colleen Murphy
52e7ebfd1b Fix partition unit test
Remove unnecessary items from the `access` slice in one of the unit
tests. Each item in `access` corresponds to one request in `apiOps`, so
they should have the same number of elements.
2023-09-20 13:58:37 -07:00