Norman Condition automatically generate condition information based on
the error the handler would return in Condition.Do function.
handler function usually return 2 types of error. the error can be
ignored and the error can not be ignored.
According to current implementation, Condition.Do function generate
condition with error state even if handler return error can be ignored.
The error that can be ignored should be ignored in the context of
condition as well.
So this commit introduce new field which is Reason to ForgetError so
that the developer can put special reason other than Error when
ForgetError is expected to happen by expected procedure like
provisioning and wait for something and Norman Condintion respect to
this field when try to generate condition information based on error
This solution will help us to fix this rancher bug
https://github.com/rancher/rancher/issues/15907
With the built in prom. metrics provider, the k8s machinery doesnt
deregister metrics when controllers are removed. So over time as
things like clusters are created or removed the metrics are not
cleaned up. The metrics types for the cache and queue are also
very large. They can take ~1GB of RAM in a 100 cluster setup.
Also, Rancher is not exposing these stats so they are unobservable.
Some worload controllers need to watch resoruces in the mangement plane
and react to them. But, they should only react to resources that
correspond to their cluster. This adds framework support for that.
