From 63436ab4a3c663e4dd54e1094dbed70857ad7d87 Mon Sep 17 00:00:00 2001 From: Renaud Gaubert Date: Fri, 21 Sep 2018 16:25:33 +0200 Subject: [PATCH 1/2] Renamed pluginwatcher README to README.md --- pkg/kubelet/util/pluginwatcher/{README => README.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename pkg/kubelet/util/pluginwatcher/{README => README.md} (100%) diff --git a/pkg/kubelet/util/pluginwatcher/README b/pkg/kubelet/util/pluginwatcher/README.md similarity index 100% rename from pkg/kubelet/util/pluginwatcher/README rename to pkg/kubelet/util/pluginwatcher/README.md From 79056292aa2f04a862119b04517ca65da1bdcb6e Mon Sep 17 00:00:00 2001 From: Renaud Gaubert Date: Fri, 21 Sep 2018 16:26:04 +0200 Subject: [PATCH 2/2] Update pluginwatcher doc --- pkg/kubelet/util/pluginwatcher/README.md | 73 +++++++++++++++++++----- 1 file changed, 59 insertions(+), 14 deletions(-) diff --git a/pkg/kubelet/util/pluginwatcher/README.md b/pkg/kubelet/util/pluginwatcher/README.md index c8b6cc28440..1b68c9c5079 100644 --- a/pkg/kubelet/util/pluginwatcher/README.md +++ b/pkg/kubelet/util/pluginwatcher/README.md @@ -1,26 +1,47 @@ +# Plugin Registration Service + This folder contains a utility, pluginwatcher, for Kubelet to register different types of node-level plugins such as device plugins or CSI plugins. It discovers plugins by monitoring inotify events under the directory returned by -kubelet.getPluginsDir(). Lets refer this directory as PluginsSockDir. -For any discovered plugin, pluginwatcher issues Registration.GetInfo grpc call -to get plugin type, name and supported service API versions. For any registered plugin type, -pluginwatcher calls the registered callback function with the received plugin -name, supported service API versions, and the full socket path. The Kubelet -component that receives this callback can acknowledge or reject the plugin -according to its own logic, and use the socket path to establish its service -communication with any API version supported by the plugin. +kubelet.getPluginsDir(). We will refer to this directory as PluginsDir. + +Plugins are expected to implement the gRPC registration service specified in +pkg/kubelet/apis/pluginregistration/v*/api.proto. + +## Plugin Discovery + +The pluginwatcher service will discover plugins in the PluginDir when they +place a socket in that directory or, at Kubelet start if the socket is already +there. + +This socket filename should not start with a '.' as it will be ignored. + + +## gRPC Service Lifecycle + +For any discovered plugin, kubelet will issue a Registration.GetInfo gRPC call +to get plugin type, name, endpoint and supported service API versions. + +Kubelet will then go through a plugin initialization phase where it will issue +Plugin specific calls (e.g: DevicePlugin::GetDevicePluginOptions). + +Once Kubelet determines that it is ready to use your plugin it will issue a +Registration.NotifyRegistrationStatus gRPC call. + +If the plugin removes its socket from the PluginDir this will be interpreted +as a plugin Deregistration + + +## gRPC Service Overview Here are the general rules that Kubelet plugin developers should follow: -- Run as 'root' user. Currently creating socket under PluginsSockDir, a root owned directory, requires - plugin process to be running as 'root'. - -- Implements the Registration service specified in - pkg/kubelet/apis/pluginregistration/v*/api.proto. +- Run plugin as 'root' user. Currently creating socket under PluginsDir, a root owned + directory, requires plugin process to be running as 'root'. - The plugin name sent during Registration.GetInfo grpc should be unique for the given plugin type (CSIPlugin or DevicePlugin). -- The socket path needs to be unique within one directory, in normal case, +- The socket path needs to be unique within one directory, in normal case, each plugin type has its own sub directory, but the design does support socket file under any sub directory of PluginSockDir. @@ -32,3 +53,27 @@ Here are the general rules that Kubelet plugin developers should follow: - For an example plugin implementation, take a look at example_plugin.go included in this directory. + + +# Kubelet Interface + +For any kubelet components using the pluginwatcher module, you will need to +implement the PluginHandler interface defined in the types.go file. + +The interface is documented and the implementations are registered with the +pluginwatcher module in kubelet.go by calling AddHandler(pluginType, handler). + + +The lifecycle follows a simple state machine: + + Validate -> Register -> DeRegister + ^ + + | | + +--------- + + +The pluginwatcher calls the functions with the received plugin name, supported +service API versions and the endpoint to call the plugin on. + +The Kubelet component that receives this callback can acknowledge or reject +the plugin according to its own logic, and use the socket path to establish +its service communication with any API version supported by the plugin.