Merge pull request #61479 from hyperbolic2346/mwilson/api-server-clear-client-ca

Automatic merge from submit-queue (batch tested with PRs 61195, 61479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Clearing out the client-ca-file option

**What this PR does / why we need it**:
kubernetes-master charm wouldn't clear the ca-client-file snap setting. We haven't used it for a while, but since it wasn't clearing it out any old deploys that updated would still have it set. This change will start clearing it.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/515
**Special notes for your reviewer**:

**Release note**:

```release-note
kubernetes-master charm now properly clears the client-ca-file setting on the apiserver snap
```

cluster/juju/layers/kubernetes-master/reactive/kubernetes_master.py

@@ -1076,6 +1076,7 @@ def configure_kubernetes_service(service, base_args, extra_args_key):
     args = {}
     for arg in prev_args:
         # remove previous args by setting to null
+        # note this is so we remove them from the snap's config
         args[arg] = 'null'
     for k, v in base_args.items():
         args[k] = v
@@ -1099,6 +1100,14 @@ def configure_apiserver(etcd_connection_string, leader_etcd_version):
     server_cert_path = layer_options.get('server_certificate_path')
     server_key_path = layer_options.get('server_key_path')
 
+    # at one point in time, this code would set ca-client-cert,
+    # but this was removed. This was before configure_kubernetes_service
+    # kept track of old arguments and removed them, so client-ca-cert
+    # was able to hang around forever stored in the snap configuration.
+    # This removes that stale configuration from the snap if it still
+    # exists.
+    api_opts['client-ca-file'] = 'null'
+    
     if is_privileged():
         api_opts['allow-privileged'] = 'true'
         set_state('kubernetes-master.privileged')