Priority is normally set by the admission controller when missing.
For static pods there seems to be some corner cases, as, if priority
is not explicitly set, Graceful Node Shutdown will shutdown
static pods with the other pods that have priority 0 or unset.
Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
Add PatchStaticPod() in staticpod/utils.go
Apply patches to static Pods in:
- phases/controlplane/CreateStaticPodFiles()
- phases/etcd/CreateLocalEtcdStaticPodManifestFile() and
CreateStackedEtcdStaticPodManifestFile()
Add unit tests and update Bazel.
While `ClusterStatus` will be maintained and uploaded, it won't be
used by the internal `kubeadm` logic in order to determine the etcd
endpoints anymore.
The only exception is during the first upgrade cycle (`kubeadm upgrade
apply`, `kubeadm upgrade node`), in which we will fallback to the
ClusterStatus to let the upgrade path add the required annotations to
the newly created static pods.
kubeadm deploys the apiserver, controller-manager and the scheduler
using liveness probes.
The bind-address option is used to configure the probe address, in
case this is configured with an unspecified address, the probe
will fail. When using an unspecified address the probe host field is
left empty, otherwise the bind-address is used.
kubeadm removed the deprecated "--address" flag for controller-manager
and scheduler in favor of "--bind-address"
We should use bind-address to configure the manifest probe addresses.
kubeadm always use the IPv4 localhost address by defaultA for etcd
The probe hostname is obtained before the generation of the etcd
parameters, so it can't detect the right IP familiy for the
host of the probe.
This causes that with IPv6 clusters doesn't work because the probe
uses the IPv4 localhost address.
This patchs configures the right localhost address based on the used
AdvertiseAddress IP family.
Etcd v3.3.0 added the --listen-metrics-urls flag which allows specifying
addition URLs to the already present /health and /metrics endpoints.
While /health and /metrics are enabled for URLS defined with
--listen-client-urls (v3+ ?) they do require HTTPS.
Replace the present etcdctl based liveness probe with a standard HTTP
GET v1.Probe that connects to http://127.0.0.1:2381/health.
These endpoints are not reachable from the outside and only available
for localhost connections.
Remove the deprecated `scheduler.alpha.kubernetes.io/critical-pod` pod annotation and use
the `priorityClassName` first class attribute instead, setting all master components to
`system-cluster-critical`.
For historical reasons InitConfiguration is used almost everywhere in kubeadm
as a carrier of various configuration components such as ClusterConfiguration,
local API server endpoint, node registration settings, etc.
Since v1alpha2, InitConfiguration is meant to be used solely as a way to supply
the kubeadm init configuration from a config file. Its usage outside of this
context is caused by technical dept, it's clunky and requires hacks to fetch a
working InitConfiguration from the cluster (as it's not stored in the config
map in its entirety).
This change is a small step towards removing all unnecessary usages of
InitConfiguration. It reduces its usage by replacing it in some places with
some of the following:
- ClusterConfiguration only.
- APIEndpoint (as local API server endpoint).
- NodeRegistrationOptions only.
- Some combinations of the above types, or if single fields from them are used,
only those field.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Added new alpha command to pivot to self hosted
Removed slelfhosting upgrade ability
Added warning message to self hosted pivot
added certs in secrets flag to new selfhosting comand
In v1alpha3's, control plane component config options were nested directly into
the ClusterConfiguration structure. This is cluttering the config structure and
makes it hard to maintain. Therefore the control plane config options must be
separated into different substructures in order to graduate the format to beta.
This change does the following:
- Introduces a new structure called ControlPlaneComponent, that contains fields
common to all control plane component types. These are currently extra args
and extra volumes.
- Introduce a new structure called APIServer that contains
ControlPlaneComponent and APIServerCertSANs field (from ClusterConfiguration)
- Replace all API Server, Scheduler and Controller Manager options in
ClusterConfiguration with APIServer, ControllerManager and Scheduler fields
of APIServer and ControlPlaneComponent types.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Order of Volumes and VolumeMounts in the pod objects created by
kubeadm is undefined as they're represended as maps in the
controlPlaneHostPathMounts struct.
This influences 'kubeadm upgrade' logic in a way that even when
manifest of the component is not changed kubeadm tries to upgrade
it because most of the time current and new pods are not equal
due to the different order of Volumes and VolumeMounts.
For example 'kubeadm apply diff' almost always shows difference
in Volumes and VolumeMounts because of this:
volumeMounts:
+ - mountPath: /etc/kubernetes/pki
+ name: k8s-certs
+ readOnly: true
- mountPath: /etc/ssl/certs
name: ca-certs
+ readOnly: true
+ - mountPath: /etc/pki
+ name: etc-pki
+ readOnly: true
+ - mountPath: /usr/share/ca-certificates
+ name: usr-share-ca-certificates
readOnly: true
- mountPath: /etc/ca-certificates
name: etc-ca-certificates
readOnly: true
- - mountPath: /etc/pki
- name: etc-pki
- readOnly: true
- - mountPath: /etc/kubernetes/pki
- name: k8s-certs
- readOnly: true
- - mountPath: /usr/share/ca-certificates
- name: usr-share-ca-certificates
- readOnly: true
Sorting Volumes and VolumeMounts should fix this issue and help
to avoid unnecessary upgrades.
When doing upgrades kubeadm generates new manifest and
waits until kubelet restarts correspondent pod.
However, kubelet won't restart pod if there are no changes
in the manifest. That makes kubeadm stuck waiting for
restarted pod.
Skipping upgrade if new component manifest is the same as
current manifest should solve this.
Fixes: kubernetes/kubeadm#1054
