Merge pull request #63894 from dims/bump-grpc-max-message-size-for-docker-service

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump grpc max message size for docker service

**What this PR does / why we need it**:
When we have a lot of containers, we run into the limit in grpc ( https://github.com/grpc/grpc-go/blob/master/clientconn.go#L118 )

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #63858

**Special notes for your reviewer**:
In https://github.com/kubernetes/kubernetes/pull/63977 we fixed the send and receive sizes on the client side. we should fix the docker service too 

**Release note**:

```release-note
NONE
```

pkg/kubelet/dockershim/remote/docker_server.go

@@ -28,6 +28,10 @@ import (
 	"k8s.io/kubernetes/pkg/util/interrupt"
 )
 
+// maxMsgSize use 8MB as the default message size limit.
+// grpc library default is 4MB
+const maxMsgSize = 1024 * 1024 * 8
+
 // DockerServer is the grpc server of dockershim.
 type DockerServer struct {
 	// endpoint is the endpoint to serve on.
@@ -60,7 +64,10 @@ func (s *DockerServer) Start() error {
 		return fmt.Errorf("failed to listen on %q: %v", s.endpoint, err)
 	}
 	// Create the grpc server and register runtime and image services.
-	s.server = grpc.NewServer()
+	s.server = grpc.NewServer(
+		grpc.MaxRecvMsgSize(maxMsgSize),
+		grpc.MaxSendMsgSize(maxMsgSize),
+	)
 	runtimeapi.RegisterRuntimeServiceServer(s.server, s.service)
 	runtimeapi.RegisterImageServiceServer(s.server, s.service)
 	go func() {