1
0
mirror of https://github.com/rancher/steve.git synced 2025-07-30 22:15:10 +00:00

Update the READMEs to cover SQL-based caching. (#693)

This commit is contained in:
Eric Promislow 2025-06-26 09:29:51 -07:00 committed by GitHub
parent 952305e096
commit c782fea615
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 83 additions and 10 deletions

View File

@ -117,7 +117,9 @@ Example, filtering by object name:
/v1/{type}?filter=metadata.name=foo
```
if a target value is surrounded by single-quotes, it succeeds only on an exact match:
When SQLite caching is not enabled, matching works this way:
If a target value is surrounded by single-quotes, it succeeds only on an exact match:
Example, filtering by object name:
@ -133,6 +135,73 @@ Example, filtering by object name:
/v1/{type}?filter=metadata.name="can-be-a-substri"
```
When SQLite caching is enabled, equality is slightly different from non-sql-supported matching.
Equality can be specified with either one or two '=' signs.
The following matches objects called either 'cat' or 'cows':
```
filter=metadata.name=cat,metadata.name==cows
```
The following matches objects whose names contain either the substring 'cat' or 'cows':
```
filter=metadata.name~cat,metadata.name~cows
```
For example, this will match an object with `metadata.name=cowcatcher`
Set membership is done with the `in` operator:
```
filter=metadata.name in (cat, cows)
```
When called via `http` the spaces will need to be encoded either as `+` or `%20`.
There are negative forms of the above operators:
```
filter=metadata.name!=dog # no dogs allowed
filter=metadata.name!~x # skip any names containing an 'x'
filter=metadata.name notin (goldfish, silverfish) # ignore these
```
Labels can be tested with the implicit "EXISTS" operator:
```
filter=metadata.labels[cattle.io.fences/wooden]
```
This will select any objects that have the specified label. Negate this test by
preceding it with a `!`:
```
filter=!metadata.labels[cattle.io.fences/bamboo]
```
Existence tests only work for `metadata.labels`.
If you need to do a numeric computation, you can use the `<` and `>` operators.
```
filter=metadata.fields[3]>10&metadata.fields[3]<20
```
This is specific to a particular kind of Kubernetes object.
Finally, most values need to conform to specific syntaxes. But if the VALUE in an
expression contains unusual characters, you can quote the value with either single
or double quotes:
```
filter=metadata.name="oxford,metadata.labels.comma"
```
Without the quotes, the expression would be finding either objects called `oxford`,
or that have the label "comma", which is very different from objects called `oxford,metadata.labels.comma`.
One filter can list multiple possible fields to match, these are ORed together:
```
@ -160,6 +229,10 @@ item is included in the list.
/v1/{type}?filter=spec.containers.image=alpine
```
When SQLite caching is enabled, multiple values are stored separated by "or-bars" (`|`),
like `abc|def|ghi`. You'll need to use the partial-match operator `~` to match one member,
like `/v1/{type}?filter=spec.containers.image ~ ghi`.
**If SQLite caching is enabled** (`server.Options.SQLCache=true`),
filtering is only supported for a subset of attributes:
- `id`, `metadata.name`, `metadata.namespace`, `metadata.state.name`, and `metadata.timestamp` for any resource kind

View File

@ -27,8 +27,8 @@ ListOptions includes the following:
* Match filters for indexed fields. Filters are for specifying the value a given field in an object should be in order to
be included in the list. Filters can be set to equals or not equals. Filters can be set to look for partial matches or
exact (strict) matches. Filters can be OR'd and AND'd with one another. Filters only work on fields that have been indexed.
* Primary field and secondary field sorting order. Can choose up to two fields to sort on. Sort order can be ascending
or descending. Default sorting is to sort on metadata.namespace in ascending first and then sort on metadata.name.
* Sort order can be ascending
or descending. Default sorting is to sort on metadata.namespace in ascending first and then sort on metadata.name. There can be any number of sort directives, comma-separated in a single `sort=` directive. Put a minus sign (`-`) before a field to sort DESC on that field (e.g. `sort=-metadata.namespace,metadata.name` sorts on the namespaces in descending order, and the names within each namespace in default ascending order).
* Page size to specify how many items to include in a response.
* Page number to specify offset. For example, a page size of 50 and a page number of 2, will return items starting at
index 50. Index will be dependent on sort. Page numbers start at 1.
@ -95,12 +95,17 @@ intended to be used as a way of enforcing RBAC.
## Technical Information
### SQL Tables
There are three tables that are created for the ListOption informer:
There are five tables that are created for the ListOption informer:
* object table - this contains objects, including all their fields, as blobs. These blobs may be encrypted.
* fields table - this contains specific fields of value for objects. These are specified on informer create and are fields
that it is desired to filter or order on.
* indices table - the indices table stores indexes created and objects' values for each index. This backs the generic indexer
that contains the functionality needed to conform to cache.Indexer.
* labels table - stores any labels created for each object.
* events table - stores any events related to each object. Fields include the revision number ("rv"), event type,
and a binary representation of the event. By default all events are stored, but only the *n* most recent events can be retained by
invoking `factory.NewCacheFactory(opts.SQLCacheFactoryOptions)` with `opts.SQLCacheFactoryOptions.DefaultMaximumEventsCount` set to the desired value.
It looks like that value currently isn't configurable from an outside config file or mechanism, but it probaby should be.
### SQLite Driver
There are multiple SQLite drivers that this package could have used. One of the most, if not the most, popular SQLite golang
@ -136,17 +141,12 @@ have the following indexes by default:
### ListOptions Behavior
Defaults:
* Sort.PrimaryField: `metadata.namespace`
* Sort.SecondaryField: `metadata.name`
* Sort.PrimaryOrder: `ASC` (ascending)
* Sort.SecondaryOrder: `ASC` (ascending)
* Sort `metadata.namespace,metadata.name` (both ASC)
* All filters have partial matching set to false by default
There are some uncommon ways someone could use ListOptions where it would be difficult to predict what the result would be.
Below is a non-exhaustive list of some of these cases and what the behavior is:
* Setting Pagination.Page but not Pagination.PageSize will cause Page to be ignored
* Setting Sort.SecondaryField only will sort as though it was Sort.PrimaryField. Sort.SecondaryOrder will still be applied
and Sort.PrimaryOrder will be ignored
### Writing Secure Queries
Values should be supplied to SQL queries using placeholders, read [Avoiding SQL Injection Risk](https://go.dev/doc/database/sql-injection). Any other portions