Production-Grade Container Scheduling and Management
Go to file
Kubernetes Submit Queue cc557e61cc Merge pull request #51473 from bboreham/cadvisor-consistent-labels
Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Fix inconsistent Prometheus cAdvisor metrics

**What this PR does / why we need it**:

We need this because otherwise kubelet is exposing different sets of Prometheus metrics that randomly include or do not include containers.

See also https://github.com/google/cadvisor/issues/1704; quoting here:

Prometheus requires that all metrics in the same family have the same labels, so we arrange to supply blank strings for missing labels

The function `containerPrometheusLabels()` conditionally adds various metric labels from container labels - pod name, image, etc. However, when it receives the metrics, Prometheus [checks](https://github.com/prometheus/client_golang/blob/master/prometheus/registry.go#L665) that all metrics in the same family have the same label set, and [rejects](https://github.com/prometheus/client_golang/blob/master/prometheus/registry.go#L497) those that do not.

Since containers are collected in (somewhat) random order, depending on which kind is seen first you get one set of metrics or the other.

Changing the container labels function to always add the same set of labels, adding `""` when it doesn't have a real value, eliminates the issue in my testing.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #50151

**Special notes for your reviewer**:

I have made the same fix in two places.  I am 98% sure the one in `cadvisor_linux.go` isn't used and indeed cannot be used, but have not gone fully down that rabbit-hole.

**Release note**:
```release-note
Fix inconsistent Prometheus cAdvisor metrics
```
2017-08-29 02:22:16 -07:00
.github
api Merge pull request #50435 from NickrenREN/localstorage-downwardapi 2017-08-29 02:22:13 -07:00
build Merge pull request #50717 from cblecker/git-util-func 2017-08-28 22:07:01 -07:00
cluster Merge pull request #51471 from ixdy/gcloud-compute-managed-list 2017-08-29 02:22:06 -07:00
cmd Merge pull request #51415 from jpbetz/timeout-param-minimal 2017-08-29 01:17:40 -07:00
docs Merge pull request #50435 from NickrenREN/localstorage-downwardapi 2017-08-29 02:22:13 -07:00
examples Merge pull request #49850 from m1093782566/service-session-timeout 2017-08-25 20:43:30 -07:00
federation Merge pull request #50435 from NickrenREN/localstorage-downwardapi 2017-08-29 02:22:13 -07:00
Godeps Bumped gRPC version to 1.3.0 2017-08-23 09:55:55 -07:00
hack Merge pull request #50932 from yguo0905/stats-cadvisor 2017-08-29 01:17:29 -07:00
logo
pkg Merge pull request #51473 from bboreham/cadvisor-consistent-labels 2017-08-29 02:22:16 -07:00
plugin Merge pull request #50805 from bsalamat/preemption_metacompute 2017-08-28 05:11:19 -07:00
staging Merge pull request #50435 from NickrenREN/localstorage-downwardapi 2017-08-29 02:22:13 -07:00
test Make threshold for glbc mem-usage scale with nodes in density test 2017-08-28 13:24:24 +02:00
third_party Use buildozer to remove deprecated automanaged tags 2017-08-11 09:31:50 -07:00
translations Italian translation 2017-08-23 08:45:26 +02:00
vendor Regenerate the vendor/BUILD file 2017-08-24 14:07:21 -07:00
.bazelrc
.generated_files
.gitattributes
.gitignore
.kazelcfg.json
BUILD.bazel
CHANGELOG.md Update CHANGELOG.md for v1.8.0-alpha.3. 2017-08-23 16:18:55 -07:00
code-of-conduct.md
CONTRIBUTING.md
labels.yaml
LICENSE
Makefile
Makefile.generated_files
OWNERS Fix my incorrect username in #46649 2017-08-10 11:59:54 -07:00
OWNERS_ALIASES add sig leads to owners-aliases 2017-08-23 14:35:20 -07:00
README.md
SUPPORT.md Add a SUPPORT.md file for github 2017-08-11 14:42:36 -04:00
Vagrantfile
WORKSPACE

Kubernetes

Submit Queue Widget GoDoc Widget


Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications.

Kubernetes builds upon a decade and a half of experience at Google running production workloads at scale using a system called Borg, combined with best-of-breed ideas and practices from the community.

Kubernetes is hosted by the Cloud Native Computing Foundation (CNCF). If you are a company that wants to help shape the evolution of technologies that are container-packaged, dynamically-scheduled and microservices-oriented, consider joining the CNCF. For details about who's involved and how Kubernetes plays a role, read the CNCF announcement.


To start using Kubernetes

See our documentation on kubernetes.io.

Try our interactive tutorial.

Take a free course on Scalable Microservices with Kubernetes.

To start developing Kubernetes

The community repository hosts all information about building Kubernetes from source, how to contribute code and documentation, who to contact about what, etc.

If you want to build Kubernetes right away there are two options:

You have a working Go environment.
$ go get -d k8s.io/kubernetes
$ cd $GOPATH/src/k8s.io/kubernetes
$ make
You have a working Docker environment.
$ git clone https://github.com/kubernetes/kubernetes
$ cd kubernetes
$ make quick-release

If you are less impatient, head over to the developer's documentation.

Support

If you need support, start with the troubleshooting guide and work your way through the process that we've outlined.

That said, if you have questions, reach out to us one way or another.

Analytics