github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-08-25 19:43:22 +00:00
Code Issues Projects Releases Wiki Activity

ARM client: survive empty response and error

Browse Source 
We're seeing legacy-cloud-providersazure/clients
(in our case, the synchronized copy used by cluster-autoscaler)
segfaulting under heavy pressure and ARM throttling, like so:

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x1b595b5]
cluster-autoscaler-all-79b9478bf5-cgkg8 cluster-autoscaler
goroutine 82 [running]:
k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/legacy-cloud-providers/azure/clients/armclient.(*Client).Send(0xc00052f520, 0x3b6dc40, 0xc000937440, 0xc000933f00, 0x0, 0x0)
     /home/jb/go/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/legacy-cloud-providers/azure/clients/armclient/azure_armclient.go:122 +0xb5
```

Reason is the ARM client expects `sendRequest()` to return either
a non nil *retry.Error, or a non nil *http.Response, while both
can be nil.
This commit is contained in:
Benjamin Pineau 2020-08-18 13:07:18 +02:00
parent fd74333a97
commit d16eee0fee
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
staging/src/k8s.io/legacy-cloud-providers/azure/clients/armclient/azure_armclient.go
View File

@ -109,6 +109,11 @@ func (c *Client) sendRequest(ctx context.Context, request *http.Request) (*http.
request,
retry.DoExponentialBackoffRetry(&sendBackoff),
)
if response == nil && err == nil {
return response, retry.NewError(false, fmt.Errorf("Empty response and no HTTP code"))
}
return response, retry.GetError(response, err)
}