kubernetes/test/images/agnhost

Agnhost

Overview

There are significant differences between Linux and Windows, especially in the way something can be obtained or tested. For example, the DNS suffix list can be found in /etc/resolv.conf on Linux, but on Windows, such file does not exist, the same information could retrieved through other means. To combat those differences, agnhost was created.

agnhost is an extendable CLI that behaves and outputs the same expected content, no matter the underlying OS. The name itself reflects this idea, being a portmanteau word of the words agnost and host.

The image was created for testing purposes, reducing the need for having different test cases for the same tested behaviour.

Usage

The agnhost binary has several subcommands which are can be used to test different Kubernetes features; their behaviour and output is not affected by the underlying OS.

For example, let's consider the following pod.yaml file:

    apiVersion: v1
    kind: Pod
    metadata:
      name: test-agnhost
    spec:
      containers:
      - args:
        - dns-suffix
        image: gcr.io/kubernetes-e2e-test-images/agnhost:2.1
        name: agnhost
      dnsConfig:
        nameservers:
        - 1.1.1.1
        searches:
        - resolv.conf.local
      dnsPolicy: None

After we've used it to create a pod:

    kubectl create -f pod.yaml

We can then check the container's output to see what is DNS suffix list the Pod was configured with:

    kubectl logs pod/test-agnhost

The output will be resolv.conf.local, as expected. Alternatively, the Pod could be created with the pause argument instead, allowing us execute multiple commands:

    kubectl exec test-agnhost -- /agnhost dns-suffix
    kubectl exec test-agnhost -- /agnhost dns-server-list

The agnhost binary is a CLI with the following subcommands:

dns-server-list

It will output the host's configured DNS servers, separated by commas.

Usage:

    kubectl exec test-agnhost -- /agnhost dns-server-list

dns-suffix

It will output the host's configured DNS suffix list, separated by commas.

Usage:

    kubectl exec test-agnhost -- /agnhost dns-suffix

etc-hosts

It will output the contents of host's hosts file. This file's location is /etc/hosts on Linux, while on Windows it is C:/Windows/System32/drivers/etc/hosts.

Usage:

    kubectl exec test-agnhost -- /agnhost etc-hosts

fake-gitserver

Fakes a git server. When doing git clone localhost:8000, you will clone an empty git repo named 8000 on local. You can also use git clone localhost:8000 my-repo-name to rename that repo.

Usage:

    kubectl exec test-agnhost -- /agnhost fake-gitserver

help

Prints the binary's help menu. Additionally, it can be followed by another subcommand in order to get more information about that subcommand, including its possible arguments.

Usage:

    kubectl exec test-agnhost -- /agnhost help

liveness

Starts a simple server that is alive for 10 seconds, then reports unhealthy for the rest of its (hopefully) short existence.

Usage:

    kubectl exec test-agnhost -- /agnhost liveness

logs-generator

The logs-generator subcommand is a tool to create predictable load on the logs delivery system. It generates random lines with predictable format and predictable average length. Each line can be later uniquely identified to ensure logs delivery.

Tool is parametrized with the total number of number that should be generated and the duration of the generation process. For example, if you want to create a throughput of 100 lines per second for a minute, you set total number of lines to 6000 and duration to 1 minute.

Parameters are passed through CLI flags. There are no defaults, you should always pass the flags to the container. Total number of line is parametrized through the flag --log-lines-total and duration in go format is parametrized through the flag --run-duration.

Inside the container all log lines are written to the stdout.

Each line is on average 100 bytes long and follows this pattern:

2000-12-31T12:59:59Z <id> <method> /api/v1/namespaces/<namespace>/endpoints/<random_string> <random_number>

Where <id> refers to the number from 0 to total_lines - 1, which is unique for each line in a given run of the container.

Examples:

docker run -i \
  gcr.io/kubernetes-e2e-test-images/agnhost:2.1 \
  logs-generator --log-lines-total 10 --run-duration 1s

kubectl run logs-generator \
  --generator=run-pod/v1 \
  --image=gcr.io/kubernetes-e2e-test-images/agnhost:2.1 \
  --restart=Never \
  -- logs-generator -t 10 -d 1s

net

The goal of this Go project is to consolidate all low-level network testing "daemons" into one place. In network testing we frequently have need of simple daemons (common/Runner) that perform some "trivial" set of actions on a socket.

Usage:

• A package for each general area that is being tested, for example nat/ will contain Runners that test various NAT features.
• Every runner should be registered via main.go:makeRunnerMap().
• Runners receive a JSON options structure as to their configuration. Run() should return the disposition of the test.

Runners can be executed into two different ways, either through the command-line or via an HTTP request:

Command-line:

    kubectl exec test-agnhost -- /agnhost net --runner <runner> --options <json>
    kubectl exec test-agnhost -- /agnhost net \
        --runner nat-closewait-client \
        --options '{"RemoteAddr":"127.0.0.1:9999"}'

HTTP server:

    kubectl exec test-agnhost -- /agnhost net --serve :8889
    kubectl exec test-agnhost -- curl -v -X POST localhost:8889/run/nat-closewait-server \
        -d '{"LocalAddr":"127.0.0.1:9999"}'

netexec

Starts a HTTP server on given TCP / UDP ports with the following endpoints:

• /: Returns the request's timestamp.
• /clientip: Returns the request's IP address.
• /dial: Creates a given number of requests to the given host and port using the given protocol, and returns a JSON with the fields responses (successful request responses) and errors ( failed request responses). Returns 200 OK status code if the last request succeeded, 417 Expectation Failed if it did not, or 400 Bad Request if any of the endpoint's parameters is invalid. The endpoint's parameters are:
  • host: The host that will be dialed.
  • port: The port that will be dialed.
  • request: The HTTP endpoint or data to be sent through UDP. If not specified, it will result in a 400 Bad Request status code being returned.
  • protocol: The protocol which will be used when making the request. Default value: http. Acceptable values: http, udp.
  • tries: The number of times the request will be performed. Default value: 1.
• /echo: Returns the given msg (/echo?msg=echoed_msg)
• /exit: Closes the server with the given code (/exit?code=some-code). The code is expected to be an integer [0-127] or empty; if it is not, it will return an error message.
• /healthz: Returns 200 OK if the server is ready, 412 Status Precondition Failed otherwise. The server is considered not ready if the UDP server did not start yet or it exited.
• /hostname: Returns the server's hostname.
• /hostName: Returns the server's hostname.
• /shell: Executes the given shellCommand or cmd (/shell?cmd=some-command) and returns a JSON containing the fields output (command's output) and error (command's error message). Returns 200 OK if the command succeeded, 417 Expectation Failed if not.
• /shutdown: Closes the server with the exit code 0.
• /upload: Accepts a file to be uploaded, writing it in the /uploads folder on the host. Returns a JSON with the fields output (containing the file's name on the server) and error containing any potential server side errors.

Usage:

    kubectl exec test-agnhost -- /agnhost netexec [--http-port <http-port>] [--udp-port <udp-port>]

nettest

A tiny web server for checking networking connectivity.

Will dial out to, and expect to hear from, every pod that is a member of the service passed in the flag --service.

Will serve a webserver on given --port, and will create the following endpoints:

• /read: to see the current state, or /quit to shut down.

• /status: to see pass/running/fail determination. (literally, it will return one of those words.)

• /write: is used by other network test pods to register connectivity.

Usage:

    kubectl exec test-agnhost -- /agnhost nettest [--port <port>] [--peers <peers>] [--service <service>] [--namespace <namespace>] [--delay-shutdown <delay>]

no-snat-test

The subcommand requires the following environment variables to be set, and they should be valid IP addresses:

• POD_IP
• NODE_IP

Serves the following endpoints on the given port (defaults to 8080).

• /whoami - returns the request's IP address.
• /checknosnat - queries ip/whoami for each provided IP (/checknosnat?ips=ip1,ip2), and if all the response bodies match the POD_IP, it will return a 200 response, 500 otherwise.

Usage:

    kubectl run test-agnhost \
      --generator=run-pod/v1 \
      --image=gcr.io/kubernetes-e2e-test-images/agnhost:2.1 \
      --restart=Never \
      --env "POD_IP=<POD_IP>" \
      --env "NODE_IP=<NODE_IP>" \
      -- no-snat-test [--port <port>]

no-snat-test-proxy

Serves the /checknosnat endpoint on the given port (defaults to 31235). The endpoint proxies the request to the given target (/checknosnat?target=target_ip&ips=ip1,ip2 -> target_ip/checknosnat?ips=ip1,ip2) and will return the same status as the status as the proxied request, or 500 on error.

Usage:

    kubectl exec test-agnhost -- /agnhost no-snat-test-proxy [--port <port>]

pause

It will pause the execution of the binary. This can be used for containers which have to be kept in a Running state for various purposes, including executing other agnhost commands.

Usage:

    kubectl exec test-agnhost -- /agnhost pause

port-forward-tester

Listens for TCP connections on a given address and port, optionally checks the data received, and sends a configurable number of data chunks, with a configurable interval between chunks.

The subcommand is using the following environment variables:

• BIND_ADDRESS (optional): The address on which it will start listening for TCP connections (default value: localhost)
• BIND_PORT: The port on which it will start listening for TCP connections.
• EXPECTED_CLIENT_DATA (optional): If set, it will check that the request sends the same exact data.
• CHUNKS: How many chunks of data to write in the response.
• CHUNK_SIZE: The expected size of each written chunk of data. If it does not match the actual size of the written data, it will exit with the exit code 4.
• CHUNK_INTERVAL: The amount of time to wait in between chunks.

Usage:

    kubectl run test-agnhost \
      --generator=run-pod/v1 \
      --image=gcr.io/kubernetes-e2e-test-images/agnhost:2.1 \
      --restart=Never \
      --env "BIND_ADDRESS=localhost" \
      --env "BIND_PORT=8080" \
      --env "EXPECTED_CLIENT_DATA='Hello there!'" \
      --env "CHUNKS=1" \
      --env "CHUNK_SIZE=10" \
      --env "CHUNK_INTERVAL=1" \
      -- port-forward-tester

webhook (Kubernetes External Admission Webhook)

The subcommand tests MutatingAdmissionWebhook and ValidatingAdmissionWebhook. After deploying it to kubernetes cluster, administrator needs to create a ValidatingWebhookConfiguration in kubernetes cluster to register remote webhook admission controllers.

TODO: add the reference when the document for admission webhook v1beta1 API is done.

Check the MutatingAdmissionWebhook and ValidatingAdmissionWebhook documentations for more information about them.

Usage:

    kubectl exec test-agnhost -- /agnhost webhook [--tls-cert-file <key-file>] [--tls-private-key-file <cert-file>]

Image

The image can be found at gcr.io/kubernetes-e2e-test-images/agnhost:2.1 for Linux containers, and e2eteam/agnhost:2.1 for Windows containers. In the future, the same repository can be used for both OSes.