* Run tests using sqlite DB in a temp directory.
I was running into write-file errors which happens when two sqlite processes try to update the DB at the same time.
* Implement and test the extractBarredValue custom SQL function.
* Explain the DB path constants better.
* Return watch error instead of logging it
The UI needs to know about watch error like `resourceversion too old` so
we need to return it.
* Sort by resourceVersion as number
The UI makes some assumption on resourceVersion. It assumes they are a
number and they are ordered by the number value. We'll want to fix this
at some point most likely but for now let's give something in a way that
UI wants.
* Remove -d suffix
After much testing, a delete of an object seems to have its own
resourceVersion so we don't need the -d suffix, we can simply use the
new resourceVersion.
* Add ListOptionIndexerOptions to create ListOptionIndexer
* Remove unused function
* Implement configurable garbage collection on ListOptionIndexer
* Propagate Steve options from Server
* Re-order SQL event hooks so events are last
* Add QueryRowContext for single line queries
* Add test case for unknown resource version
* Properly check rows and close it
* More accurate error message when context.Context is canceled
* Re-order test check
* Split RegisterAfterUpsert into two
We're going to need to be able to differentiate between Add and Update
for storing events in the _events table.
* Update mocks
* Move types related to list options and sql queries into their own package.
The problem having these in the informer package is that eventually code
in other packages will need to import `informer` only for constants or types,
but some members of the informer package may already depend on those. Best to
move type definitions into their own simpler package.
* Fix the ListOptions sort field.
Instead of making it a single array-ish field, convert it into a
true array of Sort Directives. Easier to read, less bending backwards.
* Pass the listOptions struct by reference to avoid copying.
We never update the ListOptions struct once it's created so there's
no need to pass it by value.
This might be a near-useless optimization...
* Move types related to list options and sql queries into their own package.
The problem having these in the informer package is that eventually code
in other packages will need to import `informer` only for constants or types,
but some members of the informer package may already depend on those. Best to
move type definitions into their own simpler package.
* Fix the ListOptions sort field.
Instead of making it a single array-ish field, convert it into a
true array of Sort Directives. Easier to read, less bending backwards.
* Rebasing (or human error) duplicated 'NewSortList'.
The problem having these in the informer package is that eventually code
in other packages will need to import `informer` only for constants or types,
but some members of the informer package may already depend on those. Best to
move type definitions into their own simpler package.
provisioning.cattle.io.clusters\
metadata.annotations[provisioning.cattle.io/management-cluster-display-name]
Needed to add another character to the subfieldRegex in
listoption_indexer to allow hyphens in annotation field names.
* Added more fields - part 6.
* Tweaks needed to get 'batch.cronjobs' working
* Add a clarifying comment on pulling values out of an array.
* Use the correct capitalization for the etcd-snapshot kind.
* Support sorting on metadata.labels.NAME
The key to doing this is if we want to sort on, say, `metadata.labels.foo`, we need to
search for all rows with a label of the name `foo` in all the various
join tables we create for each label the query references.
We ignore nulls by giving them lowest priority using "NULLS LAST"
("NULLS FIRST" if sorting in descending order).
* Ensure labels that are mentioned only in sort params are still selected.
If we don't do this -- say we sort on metadata.labels.foo but never
make a test on it, the sort resuilts are ignored.
* Remove extraneous debugger statements.
* Generate field names with brackets when needed.
* Stop hard-wiring complex selectors as `["field1", "field2[sub-field3]"]`
and instead represent them as a more consistent `["field1", "field2", "sub-field3"]`
* Convert all filter strings ending with square brackets to an array.
Stop special-casing 'metadata.labels[X]' and handle any query string that ends with '[...]'.
* Stop checking for pre-bracketed terms in constant field-accessor arrays.
In this commit we stop converting string arrays like
`["metadata", "labels[k8s.io/deepcode]"]` into the database field
`"metadata.labels[k8s.io/deepcode]"` and instead will do a
naive `join` to give `metadata[labels[k8s.io/deepcode]]`. The solution
is to never express the above terms in separate fields, like
`["metadata", "labels", "k8s.io/deepcode"]`. This also better reflects
the stucture of the referenced object.
* gofmt changes
* Simplify comment about 'smartJoin'.
* Create and use a synthetic watcher for non-watchable resources.
* Write unit tests for the synthetic watcher.
* Make the refresh interval for synthetic watchers configurable.
The default is to call `client.List(...)` every 5 seconds for each unwatchable GVK.
There are currently only 3 such GVKs right now so this will be informative
enough but not really noticeable.
* Pass the context into the synthetic watch func.
* Restore changes lost in rebasing.
---------
Co-authored-by: Tom Lebreux <tom.lebreux@suse.com>
Previous SQLite-related code used context.Background() and context.TODO() because it was not developed with context awareness.
This commit propagates the main Steve context so that it can be used when interacting with SQL context-aware functions.
This PR removes all production-code use of context.Background() and context.TODO() and replaces test-code use of TODO with Background.
Contributes to rancher/rancher#47825
This introduces the a `WithTransaction` function, which is then used for all transactional work in Steve.
Because `WithTransaction` takes care of all `Begin`s, `Commit`s and `Rollback`s, it eliminates the problem where forgotten open transactions can block all other operations (with long stalling and `SQLITE_BUSY` errors).
This also:
- merges together the disparate `DBClient` interfaces in one only `db.Client` interface with one unexported non-test implementation. I found this much easier to follow
- refactors the transaction package in order to make it as minimal as possible, and as close to the wrapped `sql.Tx` and `sql.Stmt` functions as possible, in order to reduce cognitive load when working with this part of the codebase
- simplifies tests accordingly
- adds a couple of known files to `.gitignore`
Credits to @tomleb for suggesting the approach: https://github.com/rancher/lasso/pull/121#pullrequestreview-2515872507
