1
0
mirror of https://github.com/rancher/steve.git synced 2025-09-01 15:37:31 +00:00
Commit Graph

324 Commits

Author SHA1 Message Date
Silvio Moioli
44a6096815 LRU cache: fix concurrent map write
Signed-off-by: Silvio Moioli <silvio@moioli.net>
2023-03-15 10:07:43 +01:00
Ricardo Weir
23ae2d3abc Revert client change
Prior, table client configs were mistakenly swapped out for regular
client configs. This caused requests using these methods to no
longer return table column data under metadata.fields. Consequently,
table data was blank in the dashboard UI. Now, the client config
swap has been reverted and table fields should be populated in the
dashboard UI.
2023-02-13 16:36:43 -07:00
Colleen Murphy
9a386b205c Fix resource counts unit tests
Increase the timeout slightly to avoid the occasional premature timeout.
Fix error message typo.
2023-01-20 14:19:05 -08:00
Michael Bolot
59a2c55e14 Fixes a bug with resource.error watch events
Watch events need to have a type defined by
k8s.io/apimachinery/pkg/watch to be properly interpreted. Fixes
a bug where this was not the case
2023-01-11 15:43:42 -06:00
Chad Roberts
65ca114ce0 Expose WarningBuffer type for external use 2023-01-03 11:32:46 -05:00
Michael Bolot
956b7351aa Initial implmentation of warning headers
Attempts to pass through warning headers which k8s returns.
Requires an update to rancher/apiserver.
2022-12-30 12:11:14 -05:00
Colleen Murphy
7565dba268 Merge pull request #63 from cmurphy/pagination-sorting-filtering
Pagination, sorting, filtering
2022-12-20 15:59:29 -08:00
Colleen Murphy
fa7fb37245 Retrieve exact revision regardless of cache state
Without this change, the cache is checked if a revision is specified,
but the revision is ignored if the value is not found in the cache. This
is a problem if by chance sequential paginated requests land on separate
pods, because a different version number may be fetched on the sequent
request and cause the caches to be inconsistent with one another. This
change ensures that `revision` is passed as `resourceVersion` to the
Kubernetes request, since `revision` has no meaning to Kubernetes.
2022-12-20 15:01:20 -08:00
Colleen Murphy
7c0228e575 Use limit=-1 to disable default list limit
The default chunk size in the partition was set to 100000. It could be
overridden as a larger or smaller number, but not disabled altogether.
This change adds the ability for users to explicitly opt out of the
limit by specifying a negative number or zero. The default behavior is
the same.
2022-12-20 15:01:20 -08:00
Colleen Murphy
b151f25581 Add secondary sort parameter
Extend the sorting functionality in the partition store to support
primary and secondary sorting criteria. Sorting parameters are specified
by a single 'sort' query and comma separated. Example:

Sort by name and creation time:

GET /v1/secrets?sort=metadata.name,metadata.creationTimestamp

Reverse sort by name, normal sort by creation time:

GET /v1/secrets?sort=-metadata.name,metadata.creationTimestamp

Normal sort by name, reverse sort by creation time:

GET /v1/secrets?sort=metadata.name,-metadata.creationTimestamp
2022-12-20 15:01:19 -08:00
Colleen Murphy
52e189b1ff Add pagination tests for partition listing 2022-12-20 15:01:19 -08:00
Colleen Murphy
b0eb50f38d Add caching to pagination
Cache filtered, sorted results for fast subsequent page retrieval.

Requests for cached queries need to include the list revision number
along with other queries. If no specific revision is requested, a new
fetch is done in order to get the latest revision. The revision is
included in the list response.

Example first request:

GET /v1/secrets?pagesize=10

Example subsequent page request:

GET /v1/secrets?pagesize=10&page=1&revision=107740
2022-12-20 15:01:19 -08:00
Colleen Murphy
9f1a27db06 Add pagination to partition store
Extend the partition store to parse page and pagesize parameters and
return a subset of list results. The total number of pages is included
in the list response.

Request an initial page:

GET /v1/secrets?pagesize=10

or

GET /v1/secrets?pagesize=10&page=1

Request subsequent pages, or arbitrary pages:

GET /v1/secrets?pagesize=10&page=37

If a page number is out of bounds, an empty list is returned.
2022-12-20 15:01:15 -08:00
Colleen Murphy
48efe7f967 Merge pull request #68 from cmurphy/rbac_partition_testing
Add tests for RBAC partitioner
2022-12-16 11:09:21 -08:00
Colleen Murphy
adecbd9122 Add sorting to partition store
Extend the partition store to parse the "sort" query parameter as a
sorting condition. Dot notation is used to denote the object field.
Preceding the key with "-" denotes descending (reverse) order.

Example sorting by name:

GET /v1/secrets?sort=metadata.name

Reverse sorting by name:

GET /v1/secrets?sort=-metadata.name

All values are converted to strings and sorted lexicographically.
2022-12-14 09:13:58 -08:00
Colleen Murphy
f8eaa11d83 Add filtering to partition store
Extend the partition store to parse query params as list filters. Filter
keys use dot notation to denote the subfield of an object to filter on.
Example:

GET /v1/secrets?filter=metadata.name=foo

Filters are ANDed together, so an object must match all filters to be
included in the list. Example:

GET /v1/secrets?filter=metadata.name=foo&filter=metadata.namespace=bar

Arrays are searched for matching items. If any item in the array
matches, the item is included in the list. Example:

GET /v1/pods?filter=spec.containers.image=alpine
2022-12-14 09:13:58 -08:00
Colleen Murphy
60d234d282 Refactor proxy store
Filtering and sorting needs to operate on unstructured data. It also
needs to operate after the parallel partitioner, higher in the store
stack. This means that the proxy Store needs to return raw, unstructured
data up to the partitioner. This change moves all conversions from
unstructured Kubernetes types to apiserver objects up from the proxy
store into the partitioner.
2022-12-14 09:13:58 -08:00
Colleen Murphy
1d4a130974 Add unit tests for partition listing 2022-12-14 09:13:58 -08:00
Michael Bolot
8fdf67a444 Merge pull request #58 from MbolotSuse/count-diff
Changing count watch to only return changed counts
2022-12-14 10:15:18 -06:00
Michael Bolot
c8fb653ca3 Bump dynamic listener to v0.3.5
Bumps dynamic listener to v0.3.5. Adds a small arg change to
account for this version bump which addresses an issue caused by
the new version
2022-12-13 16:12:33 -06:00
Colleen Murphy
23fdd030e6 Add tests for RBAC partitioner 2022-12-13 10:09:30 -08:00
Michael Bolot
37ee7bf3bd Merge pull request #67 from MbolotSuse/view-fix
Changing view links
2022-12-09 13:27:05 -06:00
Michael Bolot
fe51b08681 Tests for "Fixing view link for management crds" 2022-12-06 09:04:56 -06:00
Michael Bolot
b70a8d0548 Fixing view link for management crds 2022-12-06 09:04:31 -06:00
Kevin Joiner
e0480051d9 Merge pull request #61 from KevinJoiner/schema-wather
Updates Schemas watch logic.
2022-12-02 10:29:04 -05:00
Colleen Murphy
b16d502fc7 docs: Add docs for partition and proxy store 2022-10-31 10:34:41 -07:00
Kevin Joiner
72ab913c4c Updates Schemas watch logic.
Updated logic used to decide when and what schema events are sent
during a watch.
2022-10-27 22:12:39 -04:00
Colleen Murphy
ea61c2187a cleanup: Remove unused namespace constraint
The use of AddNamespaceConstraint was removed in e35b8304 of
rancher/rancher, so there is no possibility of there being a namespace
constraint in the request context. Remove the unused function and the
unused codepath from the rbac store.
2022-10-27 14:27:13 -07:00
Colleen Murphy
475f311d0b cleanup: Remove unused switchstore store
Usage of the switchstore store was removed when the clusters resource
was removed in dcea1db.
2022-10-27 14:27:13 -07:00
Colleen Murphy
70ea282861 cleanup: Fix API root header
The header defined in RFC 9110[1] is "Accept", not "Accepts". This
change corrects the route filter. Since this API is not documented and
the header was plainly incorrect, no attempt is made at backwards
compatibility.

[1] https://www.rfc-editor.org/rfc/rfc9110.html#name-accept
2022-10-27 14:27:13 -07:00
Colleen Murphy
18afa8868f cleanup: Remove unused function
HandlerFromConfig was not used in steve or rancher.
2022-10-27 14:27:13 -07:00
Colleen Murphy
95da447d90 cleanup: remove duplicate apiserver import
Remove duplicate import and make aliasing of other apiserver imports
consistent throughout steve.
2022-10-27 14:27:13 -07:00
Michael Bolot
b73cc57b20 Adding validate phase to the CI
Adds a validate phase to the CI which runs a linter. Also fixes
linter issues discovered during the initial run
2022-10-14 15:21:17 -05:00
Michael Bolot
5cb45b9dcd Adding test for sending counts on diffs 2022-10-03 16:19:31 -05:00
Michael Bolot
2eb41a9db7 Only return changed counts, and send initial count
Three main changes:
- When a count changes, only send that count. This makes the
count socket significantly more usable
- The count buffer will now send a count right away. This allows
a message to launch wihtout an inital buffered wait period
- The count buffer now sends messages every 5 seconds instead of 1
2022-10-03 16:12:11 -05:00
Michael Bolot
06584d207d Merge pull request #55 from MbolotSuse/access-mem-fix
Adding logic to limit number of cached schemas
2022-09-28 15:43:12 -05:00
Michael Bolot
b449b93643 Adding tests
Adds tests for:
- Cache clearing functionality
- Very basic schema generation
2022-08-26 09:28:21 -05:00
Michael Bolot
115eb31f57 Adding logic to limit number of cached schemas
Adds logic to ensure that only one schema/access set
is cached for each user. This should improve memory
consumption
2022-08-26 09:27:57 -05:00
Michael Bolot
7b5a48f871 Adding user authentication failed context value 2022-08-04 20:39:40 -05:00
Kinara Shah
3c8925cc12 wait for svc account secret to be populated with token 2022-06-28 16:25:00 -07:00
Jiaqi Luo
485ebe48a9 generate a service account token to make it work in 1.24 2022-06-24 15:47:22 -07:00
Kinara Shah
d26d34631a Merge pull request #50 from ebauman/v1.24.0 2022-06-24 15:44:43 -07:00
Colleen Murphy
11fe86ab7e Make watch timeout configurable
By default, a watch times out after 30 minutes. For debugging purposes,
it's convenient if this can be decreased. Add an environment variable
CATTLE_WATCH_TIMEOUT_SECONDS to enable setting the timeout in seconds.
2022-06-13 16:01:55 -07:00
Colleen Murphy
ada5b33d98 Return websocket error and add logging for watches
Add debug logs and send websocket messages when the watch is closed
unexpectedly.

In addition to being helpful for debugging, the dashboard specifically
looks for a `resource.error` event containing the string "too old" in
order to trigger the watch to be resynced with a refreshed revision
number.  Without this error returned, the dashboard will only see
`resource.stop` events and never change its behavior, continuing to try
to restart the watch with an incorrect resource version.
2022-06-13 15:54:40 -07:00
Eamon Bauman
3ccc837e84 webhook setup requires rest.Config now 2022-05-25 18:49:28 -05:00
Colleen Murphy
d4cfe78364 Add field filtering for resources
This change enables steve to work with three new query parameters:

"include": only include the named fields from the kubernetes object.
Subfields are denoted with ".". Subfields within arrays are ignored.
Multiple fields can be included by repeating the parameter. Example:

  GET /v1/configmaps?include=kind&include=apiVersion =>
  {
    "type": "collection",
    "links": {
      "self": "http://server/v1/configmaps"
    },
    "createTypes": {
      "configmap": "http://server/v1/configmaps"
    },
    "actions": {},
    "resourceType": "configmap",
    "revision": "327238",
    "data": [
      {
        "id": "c-m-w466b2vg/kube-root-ca.crt",
        "type": "configmap",
        "links": {
          "remove": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
          "self": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
          "update": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
          "view": "http://server/api/v1/namespaces/c-m-w466b2vg/configmaps/kube-root-ca.crt"
        },
        "apiVersion": "v1",
        "kind": "ConfigMap"
      },
    }
    ...
  }

"exclude": exclude the named fields from the kubernetes object.
Subfields are denoted with ".". Subfields within arrays are ignored.
Multiple fields can be excluded by repeating the parameter. Example:

  GET /v1/configmaps?exclude=data&exclude=metadata.managedFields =>
  {
  "type": "collection",
  "links": {
    "self": "http://server/v1/configmaps"
  },
  "createTypes": {
    "configmap": "http://server/v1/configmaps"
  },
  "actions": {},
  "resourceType": "configmap",
  "revision": "328086",
  "data": [
    {
      "id": "c-m-w466b2vg/kube-root-ca.crt",
      "type": "configmap",
      "links": {
        "remove": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
        "self": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
        "update": "http://server/v1/configmaps/c-m-w466b2vg/kube-root-ca.crt",
        "view": "http://server/api/v1/namespaces/c-m-w466b2vg/configmaps/kube-root-ca.crt"
      },
      "apiVersion": "v1",
      "kind": "ConfigMap",
      "metadata": {
        "creationTimestamp": "2022-04-11T22:05:27Z",
        "fields": [
          "kube-root-ca.crt",
          1,
          "25h"
        ],
        "name": "kube-root-ca.crt",
        "namespace": "c-m-w466b2vg",
        "relationships": null,
        "resourceVersion": "36948",
        "state": {
          "error": false,
          "message": "Resource is always ready",
          "name": "active",
          "transitioning": false
        },
        "uid": "1c497934-52cb-42ab-a613-dedfd5fb207b"
      }
    },
    ...
  }

"excludeValues": replace the values of an object with empty strings, leaving
the keys in place. Useful for showing a summary of an object with large
values, such as the data in a ConfigMap. Only works on fields that are
object. Multiple fields can have values excluded by repeating the
parameter. Example:

  GET /v1/configmaps?excludeValues=data =>
  {
    "type": "collection",
    ...
    "data": [
      {
        ...
        "data": {
          "ca.crt": ""
        },
        ...
      },
      ...
    ]
  }
2022-04-13 08:50:39 -07:00
Jake Hyde
6bb3915906 Prevent write on closed channel 2022-03-02 19:19:02 -05:00
Ricardo Weir
e9472ecc34 Add metrics 2022-02-14 17:39:37 -07:00
michelia
42c575a009 Fixed errors in obtaining object relationships
Issue: https://github.com/harvester/harvester/issues/1857
2022-01-26 19:31:03 +08:00
CMC
fea67065db feat: support k8s version less than 1.16 2021-11-25 16:16:20 +08:00