Commit Graph

97895 Commits

Author SHA1 Message Date
Artyom Lukianov
ff2a110920 memory manager: provide the new type to contain resources for each NUMA node
Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
74eeef2a0a memory manager: provide additional e2e tests
With the memory manager static policy:

- start multiple guaranteed pods and verify that pods succeeded to start
- start workload pod on each NUMA node to load the memory and start the
  pod that requested more memory than each NUMA node have, the pod should fail
  to start with the admission error, because no single NUMA node has enough
  memory to start the pod and also each NUMA node already used for single
  NUMA node allocation
  The test requires at least two NUMA nodes

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Pawel Rapacz
606fea29f5 memory manager: add e2e test to run guaranteed pod with init containers
Signed-off-by: Pawel Rapacz <p.rapacz@partner.samsung.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
f3d4ac2f7e memory manager: add basice e2e tests
Provides basic tests e2e to verify that pod succeeds
to start with MemoryManager enabled.

Verifies both MemoryManager policies and when the node has
multiple NUMA nodes it will verify the memory pinning.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
a015e4163f memory manager: rename state structs and fields
The commit rename state structs and some fields under these structs.

- NodeMap -> NUMANodeMap
- NodeState -> NUMANodeState
- NUMANodeState.Nodes -> NUMANodesState.Cells

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
aa63e5aed2 memory manager: provide an additional validation for reserved memory
Calculate the total amount of reserved memory only for NUMA nodes
that are existing under the machine.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
27c5efe8ec memory manager: fix scheme unit test
Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:10:00 +02:00
Artyom Lukianov
24be74e759 memory manager: update bazel files
Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:09:59 +02:00
Artyom Lukianov
f7845ed4e9 memory manager: provide memory manager static policy unittests
Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:09:59 +02:00
Krzysztof Wiatrzyk
d7175a831c memory manager: adding Memory Manager component unit tests
Signed-off-by: Krzysztof Wiatrzyk <k.wiatrzyk@samsung.com>
2021-02-09 01:09:59 +02:00
Pawel Rapacz
18c8a821e0 memory manager: implement GetPodTopologyHints method
It will return memory and hugepages hints for the whole pod.

Signed-off-by: Pawel Rapacz <p.rapacz@partner.samsung.com>
2021-02-09 01:09:59 +02:00
Artyom Lukianov
abb94bec51 memory manager: implement the memory manager static policy
- The `Allocate` method will try to allocate the memory according to the affinity hints
  saved under the `TopologyManager` store. If the store does not have any hints for the memory
  it will call `getDefaultHint` to get the default hint. If the affinity does not satisfy
  the memory request, it will call `extendTopologyManagerHint` to extend the topology hint to
  satisfy the memory request. Once it has the preferred hint, it will allocate the memory and
  update the the memory manager state accordingly.

- The `RemoveContainer` will release the allocated memory and update the memory manager state accordingly.

- The `GetTopologyHints` method will try to re-generate topology hints when the container already presents
  under the memory manager state. If it does not present it will call `calculateHints` to get topology hints.
  The `calculateHints` uses an approach similar to the one used under the CPU manager:
  1. If the container memory request can be satisfied by the single NUMA node, it will not allocate the memory from
     more than one NUMA node and it will set only single NUMA hints as the preferred one.
     It can affect the density, but it gives us guarantees regarding the NUMA alignment.
  2. The NUMA node used in the multi NUMA assignment can not be used in the single NUMA assignment.
     And the NUMA node used in the single NUMA assignment can not be used in the multi NUMA assignment.
  3. Only hints with NUMA node that have enough memory will be returned.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:09:59 +02:00
Artyom Lukianov
371c918e6c memory manager: add memory manager policy to defaulter and conversion files
Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:09:59 +02:00
Artyom Lukianov
afb1ae3458 memory manager: add fake memory manager
The fake memory manager needed for the unittesting.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 01:09:59 +02:00
Derek Carr
b7009a2cba Mark vishh as emeritus sig-node approver 2021-02-08 17:56:26 -05:00
Cezary Zukowski
4a64102918 memory manager: validate reserved-memory against Node Allocatable
Reserved memory of all kinds (and over all
NUMA nodes) must be equal to the values determined
by Node Allocatable feature.

Signed-off-by: Cezary Zukowski <c.zukowski@samsung.com>
2021-02-09 00:54:58 +02:00
Krzysztof Wiatrzyk
711e85af24 memory manager: adding additional tests for server.go file, for parseReservedMemoryConfig() function. 2021-02-09 00:54:58 +02:00
Artyom Lukianov
9ae499ae46 memory manager: pass memory manager flags to the container manager
Pass memory manager flags to the container manager and call all relevant memory manager
methods under the container manager.

Signed-off-by: Byonggon Chun <bg.chun@samsung.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
93accb51e4 memory manager: add memory manager flag under kubelet options and kubelet config
The commit also includes generated files after `make generated_files`.

Signed-off-by: Byonggon Chun <bg.chun@samsung.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
b95d45e803 memory manager: add new flag type BracketSeparatedSliceMapStringString
Add BracketSeparatedSliceMapStringString to parse config like the below
{numa-node=0,type=memory,limit=1Gi},{numa-node=1,type=memory,limit=1Gi}

Signed-off-by: Byonggon Chun <bg.chun@samsung.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
95f81372e2 memory manager: implement the manager interface methods
The commit adds implementation for methods:
- Start
- AddContainer
- Allocate
- RemoveContainer
- State
- GetTopologyHints

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
d0caec90e2 memory manager: add the policy interface
The commit also adds two policy skeletons:
- none
- static

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
86df524948 memory manager: provide unittest for the state package
The commit includes tests to verify the functionallity:
- to restore state from the file
- to store the state to the file
- to clean the state from old data

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
48ca6e53e6 memory manager: provide and use the checkpoint manager
The checkpoint manager provides a way to save the memory manager
`MemoryTable` both under the memory and under the state file.

Saving the `MemoryTable` under the state file can be useful when kubelet
restarted and you want to restore memory allocations for running containers.
Also, it provides a way to monitor memory allocations done by the memory manager,
and in the future, the state file content can be exposed under the pod metrics.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 00:54:58 +02:00
Artyom Lukianov
4c75be0604 memory manager: provide the skeleton for the memory manager
Provide memory manager struct and methods that should be implemented.

Signed-off-by: Artyom Lukianov <alukiano@redhat.com>
2021-02-09 00:54:58 +02:00
Stephen Heywood
c88a838282 Promote pod and service ProxyWithPath e2e test to Conformance 2021-02-09 11:50:22 +13:00
Han Kang
c3f550f3cb fix arguments too long error 2021-02-08 11:55:25 -08:00
Kubernetes Prow Robot
f266f60da9
Merge pull request #98839 from Mattias-/kubeadm-fix-config-print-placeholder
kubeadm: Fix placeholder in config print help text
2021-02-08 11:54:43 -08:00
Michelle Au
1025997abf PD CSI: disable resizer volume inuse error check as it is not necessary for PD and requires new RBACs
Change-Id: I5844eaed0b23389ac07584e8b9ee82f47df5a90e
2021-02-08 11:27:01 -08:00
Kubernetes Prow Robot
d11e475b9b
Merge pull request #98770 from pohly/e2e-mock-container-versions
test: bump CSI container versions for mock testing
2021-02-08 10:45:37 -08:00
Jacek Kaniuk
c891207ab7 Simplify leader election code 2021-02-08 19:39:42 +01:00
Kubernetes Prow Robot
0867767d6e
Merge pull request #98882 from kvaps/flatten-adminconfig
kubeadm: flatten admin.conf for cluster-info generation
2021-02-08 09:26:28 -08:00
Andrei Kvapil
9de0b2bf46
kubeadm: flatten admin.conf for cluster-info 2021-02-08 17:22:01 +01:00
Kubernetes Prow Robot
74e5080b0e
Merge pull request #97251 from navidshaikh/pr/fix-68026-test-e2e-common
Fix golint errors in test/e2e/common
2021-02-08 08:11:47 -08:00
Claudiu Belu
3945c8972c tests: Switch dockerhub registry images to promoted registry
Due to the dockerhub rate limiting, we had to find an alternative solution. We've mirrored the dockerhub
images into our own.

Additionally, our own busybox, httpd, and nginx images also have Windows support.
2021-02-08 08:01:04 -08:00
Maciej Szulik
c24b0a2a08
Promote cronjob controller v2 to beta 2021-02-08 14:54:20 +01:00
Dave Chen
06bc3b48af Improve test coverage for service affinity
Pods that are filtered by the plugin should has UID associated, otherwise all those
matching pods will be filtered out, this is the against of the spec of testcases.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2021-02-08 17:02:53 +08:00
Navid Shaikh
2e63bad5a3 Fix golint errors in test/e2e/common
- Add comments for exported types
 - Replace Uid with UID
2021-02-08 10:47:41 +05:30
Tonis Tiigi
fd01467c0c avoid hardcoding amd64 specifics on cross compiling
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-07 21:08:00 -08:00
Kubernetes Prow Robot
97d40890d0
Merge pull request #98802 from dabaooline/fix-ignorederr
fix ignored podinfo parserr
2021-02-07 17:53:11 -08:00
Abdullah Gharaibeh
880bbdad23 Graduate TTLAfterFinished to beta 2021-02-07 17:23:14 -05:00
Kubernetes Prow Robot
15a8a8ec4a
Merge pull request #97738 from knabben/deny-all-ns
Default DENY all egress from pods in the namespace
2021-02-07 12:37:12 -08:00
Amim Knabben
f1da11078a UDP protocol on connect agnhost command 2021-02-07 15:29:58 -05:00
Sai Harsha Kottapalli
a2ef69060a
fix staticcheck failures for test/e2e and test/integration 2021-02-07 23:47:23 +05:30
Kubernetes Prow Robot
fc7b785e64
Merge pull request #98818 from chewong/remove-duplicated-code
fix: remove duplicated code in image-util.sh due to merge conflict
2021-02-07 09:23:11 -08:00
Amim Knabben
189edf1e73 Default deny all egress from pods in the namespace 2021-02-07 11:55:50 -05:00
Amim Knabben
c8031e5b13 Netpol e2e test should overwrite allow with another deny 2021-02-07 11:48:20 -05:00
Claudiu Belu
6b090886de tests: Separate registry configuration for etcd image
Current gcRegistry configuration is responsible for both etcd and pause images. We should
use the upstream pause image in testing.

The etcd image doesn't have Windows support yet, so we need to have a separate configuration
for it.
2021-02-07 06:20:14 -08:00
Claudiu Belu
3e0b66b802 test images: Bump resource-consumer to 1.9
We've added Windows support to the resource-consumer image and 1.8 tag is already promoted.
We need to bump the VERSION, so we can promote the new image.
2021-02-06 23:54:25 -08:00
Angela Li
1edb548b09 Use -LiteralPath instead of -Path 2021-02-06 22:45:46 -08:00