Automatic merge from submit-queue
systemd node spec proposal
The following outlines changes that I want to make to the ```kubelet``` in order to better integrate with ```systemd``` systems, and to better isolate containers in their own ```cgroup``` based on the qos tier.
I think this is a precursor to getting more intelligent low compute resource eviction.
/cc @smarterclayton @ncdc @pmorie @dchen1107 @vishh @bgrant0607
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/17688)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Make DNS validators return error strings
Part of the larger validation PR, broken out for easier review and merge. Builds on previous PRs in the series.
Check docker's pid file, then fallback to pidof when trying to determine the pid for docker. The
latest docker RPM for RHEL changes /usr/bin/docker from an executable to a shell script (to support
/usr/bin/docker-current and /usr/bin/docker-latest). The pidof check for docker fails in this case,
so we check /var/run/docker.pid first (the default location), and fallback to pidof if that fails.
Automatic merge from submit-queue
gcr.io/google_containers/mounttest: use Stat instead of Lstat
The current ``mt.go`` implementation use ``os.Lstat`` instead of ``os.Stat`` which does not read symlinks. Since implementation of ``AtomicWriter`` (which relies on existence of symlinks), the updated implementation of secret volume using the ``AtomicWriter`` can not be tested for secret file permission. Replacing ``Lstat`` with ``Stat`` allows to read symlinks and return permissions of target file. The change affects ``--file_perm`` and ``--file_mode`` options only.
``mounttest`` image is currently used by:
##### downwardapi_volume.go
- e2e: Downward API volume
- version: 0.6
- args: --file_content, --break_on_expected_content, --retry_time, --file_content_in_loop
##### empty_dir.go
- e2e: EmptyDir volumes
- version: 0.5
- args: --file_perm, --file_perm, ...
##### host_path.go
- e2e: hostPath
- version: 0.6
- args: --file_mode, ...
##### configmap.go
- e2e: ConfigMap
- version: 0.6
- args: --file_content, --break_on_expected_content, --retry_time, --file_content_in_loop
##### service_accounts.go
- e2e: ServiceAccounts
- version: 0.2
- args: --file_content
Some of the e2e tests use at least one of the affected options. Locally, I have updated all version of mounttest images to 0.7. All e2e tests pass with the new image.
- create 100 PV, ranging from 0 to 99GB; create 1 PVC to claim 50GB. Verify only one PV is bound and rest are pending
- create 2 PVs with different access modes (RWM, RWO), 1 PVC to claim RWM PV. Verify RWM is bound and RWO is not bound.
Signed-off-by: Huamin Chen <hchen@redhat.com>
Automatic merge from submit-queue
AWS: Move enforcement of attached AWS device limit from kubelet to scheduler
Limit of nr. of attached EBS volumes to a node is now enforced by scheduler. It can be adjusted by `KUBE_MAX_PD_VOLS` env. variable there. Therefore we don't need the same check in kubelet. If the system admin wants to attach more, we should allow it.
Kubelet limit is now 650 attached volumes ('ba'..'zz').
Note that the scheduler counts only *pods* assigned to a node. When a pod is deleted and a new pod is scheduled on a node, kubelet start (slowly) detaching the old volume and (slowly) attaching the new volume. Depending on AWS speed **it may happen that more than KUBE_MAX_PD_VOLS volumes are actually attached to a node for some time!** Kubelet will clean it up in few seconds / minutes (both attach/detach is quite slow).
Fixes#22994
Automatic merge from submit-queue
kube-apiserver options should be decoupled from impls
A few months ago we refactored options to keep it independent of the
implementations, so that it could be used in CLI tools to validate
config or to generate config, without pulling in the full dependency
tree of the master. This change restores that by separating
server_run_options.go back to its own package.
Also, options structs should never contain non-serializable types, which
storagebackend.Config was doing with runtime.Codec. Split the codec out.
Fix a typo on the name of the etcd2.go storage backend.
Finally, move DefaultStorageMediaType to server_run_options.
@nikhiljindal as per my comment in #24454, @liggitt because you and I
discussed this last time
Sometimes clients send unintelligible data to the server, provide a bit
more debugging in the returned error to make it easier to pin down where
the problem is from the user side.
Provides fast transformations of int64 scaled values with overflow to
inf.Dec as needed. Attempts to reduce allocations and expensive
operations in the critical path.
Alters the public signature of Quantity to encourage more efficient use
Recycling is a long duration process and when the recycler controller is
restarted in the meantime, it should not start a new recycler pod if there is
one already running.
This means that the recycler pod must have deterministic name based on name
of the recycled PV, we then get name conflicts when creating the pod.
Two things need to be changed:
- recycler controller and recycler plugins must pass the PV.Name to place,
where the pod is created.
- create recycler pod with deterministic name and check "already exists" error.
When at it, remove useless 'resourceVersion' argument and make log messages
starting with lowercase.
Automatic merge from submit-queue
Refactor persistent volume controller
Here is complete persistent controller as designed in https://github.com/pmorie/pv-haxxz/blob/master/controller.go
It's feature complete and compatible with current binder/recycler/provisioner. No new features, it *should* be much more stable and predictable.
Testing
--
The unit test framework is quite complicated, still it was necessary to reach reasonable coverage (78% in `persistentvolume_controller.go`). The untested part are error cases, which are quite hard to test in reasonable way - sure, I can inject a VersionConflictError on any object update and check the error bubbles up to appropriate places, but the real test would be to run `syncClaim`/`syncVolume` again and check it recovers appropriately from the error in the next periodic sync. That's the hard part.
Organization
---
The PR starts with `rm -rf kubernetes/pkg/controller/persistentvolume`. I find it easier to read when I see only the new controller without old pieces scattered around.
[`types.go` from the old controller is reused to speed up matching a bit, the code looks solid and has 95% unit test coverage].
I tried to split the PR into smaller patches, let me know what you think.
~~TODO~~
--
* ~~Missing: provisioning, recycling~~.
* ~~Fix integration tests~~
* ~~Fix e2e tests~~
@kubernetes/sig-storage
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24331)
<!-- Reviewable:end -->
Fixes#15632
It should reduce the resource data after finding the pod in the pods, because perhaps no corresponding pod in the pods of the node, at this time it shouldn't reduce the resource data of the node.
Automatic merge from submit-queue
Make name validators return string slices
Part of the larger validation PR, broken out for easier review and merge. Builds on previous PRs in the series.
