Commit Graph

332 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
96ec318718 Merge pull request #59842 from ixdy/update-rules_go-02-2018
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies

**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.

**Release note**:

```release-note
NONE
```
2018-02-19 22:23:05 -08:00
Jeff Grafton
ef56a8d6bb Autogenerated: hack/update-bazel.sh 2018-02-16 13:43:01 -08:00
Aleksandra Malinowska
2d54ba3e0f Revert "add node shutdown taint" 2018-02-16 12:24:27 +01:00
Kubernetes Submit Queue
271c267fff Merge pull request #59830 from khenidak/az-ratelimit
Automatic merge from submit-queue (batch tested with PRs 59939, 59830). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Azure - ARM Read/Write rate limiting

**What this PR does / why we need it**:

Azure cloud provider currently runs with:
1. Single ARM rate limiter for both `read [put/post/delete]` and `write` operations, while ARM provide [different rates for read/write] (https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits). This causes write operation to stop even if there is available write request quotas. 
2. Cloud provider uses rate limiter's `Accept()` instead of `TryAccept()` This causes control loop to wait for prolonged tike `in case of no request quota available` for **all** requests even for those does not require ARM interaction. A case for that the `Service` control loop will wait for a prolonged time trying to create `LoadBalancer` service even though it can fail and work on the next service which is `ClusterIP`. This PR moves cloud provider tp `TryAccept()`

**Which issue(s) this PR fixes**:
Fixes # https://github.com/kubernetes/kubernetes/issues/58770

**Special notes for your reviewer**:
`n/a`

**Release note**:

```release-note
- Separate current ARM rate limiter into read/write
- Improve control over how ARM rate limiter is used within Azure cloud provider
```

cc @jackfrancis (need your help carefully reviewing this one) @brendanburns @jdumars
2018-02-15 16:43:37 -08:00
Khaled Henidak(Kal)
38a9fc33db code review: create err chan via helper 2018-02-15 20:11:40 +00:00
Kubernetes Submit Queue
27daaab224 Merge pull request #59323 from zetaab/nodetaint
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add node shutdown taint

**What this PR does / why we need it**: we need node stopped taint in order to detach volumes immediately without waiting timeout. More info in issue ticket #58635 

**Which issue(s) this PR fixes** 
Fixes #58635

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-02-15 09:52:10 -08:00
Khaled Henidak(Kal)
53036bf755 Code review + resync VMSS changes 2018-02-15 16:49:47 +00:00
Kubernetes Submit Queue
d89e64110a Merge pull request #59716 from feiskyer/vmss-disk
Automatic merge from submit-queue (batch tested with PRs 59489, 59716). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add AzureDisk support for vmss nodes

**What this PR does / why we need it**:

This PR adds AzureDisk support for vmss nodes. Changes include

- Upgrade vmss API to 2017-12-01
- Upgrade vmss clients with new version API
- Abstract AzureDisk operations for vmss and vmas
- Added AzureDisk support for vmss
- Unit tests and fake clients fix
 
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #43287

**Special notes for your reviewer**:

~~Depending on #59652 (the first two commits are from #59652).~~

**Release note**:

```release-note
Add AzureDisk support for vmss nodes
```
2018-02-14 00:14:34 -08:00
Pengfei Ni
fbc871be32 Add vmType checking in Azure disk controller common 2018-02-14 08:38:34 +08:00
Khaled Henidak(Kal)
f909859b6d fix json tag on Azure.config 2018-02-13 22:22:25 +00:00
Khaled Henidak(Kal)
9e4f1441b4 Merge branch 'master' of https://github.com/kubernetes/kubernetes into az-ratelimit 2018-02-13 20:57:05 +00:00
Khaled Henidak(Kal)
a86062c259 Configuration changes 2018-02-13 20:56:53 +00:00
Khaled Henidak(Kal)
5bf6b0fd70 WIP - create read/writer rate limiter 2018-02-13 20:05:20 +00:00
Kubernetes Submit Queue
91c783e20b Merge pull request #59739 from andyzhangx/azuredisk-initaccount
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the error prone account creation method of blob disk

**What this PR does / why we need it**:
use new account generation method for blob disk to fix the error prone account creation method of blob disk

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59738

**Special notes for your reviewer**:

**Release note**:

```
fix the error prone account creation method of azure blob disk
```

/assign @karataliu 
/sig azure
2018-02-12 19:03:37 -08:00
andyzhangx
8a7198b036 use new account generation method for blob disk
fix comments

change azureDiskSharedAccountNamePrefix var

rename sharedDiskAccountNamePrefix

use default vhd container name as "vhds"

use one commaon func: SearchStorageAccount

fix comments
2018-02-12 13:09:29 +00:00
Pengfei Ni
8c382649ba Add unit tests for mapLoadBalancerNameToVMSet 2018-02-12 16:07:11 +08:00
Pengfei Ni
d3e0280c3a Map correct vmset name for internal load balancers 2018-02-12 15:56:57 +08:00
Pengfei Ni
1976983e79 Fix unit tests for vmss 2018-02-12 14:07:05 +08:00
Pengfei Ni
890b7efb1d Add azure disk support of vmss 2018-02-12 14:07:05 +08:00
Pengfei Ni
5042cea857 Use new clients for vmss cache 2018-02-12 14:07:05 +08:00
Pengfei Ni
3b7cc3dd5e Update Azure GO SDK to v12.4.0-beta 2018-02-12 14:07:05 +08:00
Pengfei Ni
5d16067ddd Update vmss fake clients 2018-02-12 14:07:05 +08:00
Pengfei Ni
829e0946e5 Update vmss client to new version 2018-02-12 14:07:05 +08:00
Pengfei Ni
11e5399dde Abstract disk operation interfaces in VMSet 2018-02-12 14:07:05 +08:00
Kubernetes Submit Queue
69324f90e6 Merge pull request #59652 from feiskyer/vmss-cache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add generic cache for Azure VMSS

**What this PR does / why we need it**:

This PR adds a generic cache for VMSS and removes old list-based cache.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

Continue of ##58770.

**Special notes for your reviewer**:

Depends on #59520.

**Release note**:

```release-note
Add generic cache for Azure VMSS
```
2018-02-11 21:22:45 -08:00
Pengfei Ni
bde2989c7a Add unit tests for extractVmssVMName 2018-02-12 11:21:08 +08:00
Kubernetes Submit Queue
cfa6774540 Merge pull request #56557 from andyzhangx/azurefile-createaccount
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the create azure file pvc failure if there is no storage account in current resource group

**What this PR does / why we need it**:
When create an azure file PVC, there will be error if there is no storage account in current resource group.
With this PR, a storage account will be created if there is no storage account in current resource group.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56556

**Special notes for your reviewer**:
1. rephrase the code logic of `CreateFileShare` func.
```
if accountName is empty, then 
    find a storage account that matches accountType
    if no storage account found, then
        create a new account
else
    we only use user specified storage account

create a file share according to found storage account
```
2. Use func `getStorageAccountName` to get a unique storage account name by UUID, a storage account for azure file would be like `f0b2b0bd40c010112e897fa`. And in next PR, I will use this function to create storage account for azure disk, the storage account for azure disk would be like `d8f3ad8ad92000f1e1e88bd`.

**Release note**:

```
fix the create azure file pvc failure if there is no storage account in current resource group
```
/sig azure
/assign @rootfs
2018-02-11 05:03:32 -08:00
andyzhangx
aa21bef677 create storage account if necessary when create azure file pvc
use new storage account name generation method

use uuid to generate account name

change azure file account prefix

use uniqueID to generate a storage account name

fix comments

fix comments

fix comments

fix a storage account matching bug

only use UUID in getStorageAccountName func

use shorter storage account prefix for azure file

fix comments

fix comments

fix comments

fix rebase build error

rewrite CreateFileShare code logic

fix gofmt issue

fix test error

fix comments

fix a location matching bug
2018-02-11 06:29:05 +00:00
Kubernetes Submit Queue
317853c90c Merge pull request #59464 from dixudx/fix_all_typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix all the typos across the project

**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.

This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy 
/assign @fejta 

**Release note**:

```release-note
None
```
2018-02-10 22:12:45 -08:00
Pengfei Ni
f32bd6a081 Adjust unit tests for vmss 2018-02-11 11:09:22 +08:00
Pengfei Ni
4d5e7b7cfb Use generic cache for vmss 2018-02-11 11:09:22 +08:00
Kubernetes Submit Queue
f0e573d6d5 Merge pull request #59520 from feiskyer/new-cache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add generic cache for Azure VM/LB/NSG/RouteTable

**What this PR does / why we need it**:

Part of #58770. This PR adds a generic cache of Azure VM/LB/NSG/RouteTable for reducing ARM calls.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #58770

**Special notes for your reviewer**:

**Release note**:

```release-note
Add generic cache for Azure VM/LB/NSG/RouteTable
```
2018-02-10 19:06:02 -08:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Pengfei Ni
7634eacb4f Add error handling and new tests 2018-02-09 20:38:29 +08:00
liaoj
953cbb11a1 Update azure_loadbalancer.md
fix typo 
incase -> in case
selction -> selection
2018-02-09 12:52:59 +08:00
Pengfei Ni
daec2bd745 Add cache for route tables 2018-02-09 09:09:25 +08:00
Pengfei Ni
21c8a63689 Add cache for network security groups 2018-02-09 09:09:25 +08:00
Pengfei Ni
d22b6d9ebe Add cache for load balancer 2018-02-09 09:09:25 +08:00
Pengfei Ni
2badf1ff55 Add cache for virtual machines 2018-02-09 09:09:25 +08:00
Pengfei Ni
035c8da63d New unit tests for timedCache 2018-02-09 09:09:25 +08:00
Pengfei Ni
259dbf8da7 Make azure cache general for all objects 2018-02-09 09:09:25 +08:00
Jesse Haka
3cf5b172fa add node shutdown taint
shutdowned -> stopped

use shutdown everywhere

use patch in taints api call

use notimplemented in clouds use AddOrUpdateTaintOnNode

correct log text

add fake cloud

try to fix bazel

add shutdown tests

add context
2018-02-08 12:56:06 +02:00
Pengfei Ni
20de29ae6f Set instanceID to azure resource ID format while useInstanceMetadata is enabled 2018-02-08 15:12:36 +08:00
Pengfei Ni
4e4fde93a7 Add useInstanceMetadata param back in Azure cloud provider
This reverts commit bb1e797b28.
2018-02-08 12:40:35 +08:00
Walter Fender
e18e8ec3c0 Add context to all relevant cloud APIs
This adds context to all the relevant cloud provider interface signatures.
Callers of those APIs are currently satisfied using context.TODO().
There will be follow on PRs to push the context through the stack.
For an idea of the full scope of this change please look at PR #58532.
2018-02-06 12:49:17 -08:00
Kubernetes Submit Queue
87f77cd237 Merge pull request #59340 from feiskyer/fip
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Ensure public IP removed after service deleted

**What this PR does / why we need it**:

When creating many LoadBalancer services, some services may exceed Azure basic LB's FrontendIPConfiguations quota (default is 10). Public IPs are created for all services, but it is not removed after deleting the kubernetes services.

This PR fixes the problem.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59255

**Special notes for your reviewer**:

Should cherry-pick to v1.9.

**Release note**:

```release-note
Ensure Azure public IP removed after service deleted
```
2018-02-06 08:36:24 -08:00
Kubernetes Submit Queue
db23481ce8 Merge pull request #59165 from brendandburns/ut
Automatic merge from submit-queue (batch tested with PRs 59165, 59386). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Refactor and add some tests.

Improve test coverage of the Azure cloud provider.
2018-02-05 23:50:31 -08:00
Pengfei Ni
e2eaccb829 Ensure public IP removed after service deleted 2018-02-06 15:04:15 +08:00
Brendan Burns
d6bb434730 Refactor and add some tests. 2018-02-05 19:00:39 -08:00
Kubernetes Submit Queue
ffda1e2200 Merge pull request #57017 from andyzhangx/azurefile-growsize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add PV size grow feature for azure file

**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure file

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56462 

**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.

**Release note**:

```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns
2018-02-05 11:25:48 -08:00