github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-09-07 04:03:20 +00:00
Code Issues Projects Releases Wiki Activity

Organize troubleshooting sections

Browse Source
This commit is contained in:
Janet Kuo
2015-07-15 10:03:43 -07:00
parent c208c1c9a3
commit ec2b125e25
5 changed files with 87 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
docs/user-guide/application-troubleshooting.md
View File

@@ -20,7 +20,7 @@ certainly want the docs that go with that version.</h1>
<!-- END STRIP_FOR_RELEASE -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Application Troubleshooting.
# Application Troubleshooting
This guide is to help users debug applications that are deployed into Kubernetes and not behaving correctly.
This is *not* a guide for people who want to debug their cluster. For that you should check out
@@ -28,10 +28,18 @@ This is *not* a guide for people who want to debug their cluster. For that you
**Table of Contents**
<!-- BEGIN MUNGE: GENERATED_TOC -->
- [Application Troubleshooting.](#application-troubleshooting.)
- [Application Troubleshooting](#application-troubleshooting)
- [FAQ](#faq)
- [Diagnosing the problem](#diagnosing-the-problem)
- [Debugging Pods](#debugging-pods)
- [My pod stays pending](#my-pod-stays-pending)
- [My pod stays waiting](#my-pod-stays-waiting)
- [My pod is crashing or otherwise unhealthy](#my-pod-is-crashing-or-otherwise-unhealthy)
- [Debugging Replication Controllers](#debugging-replication-controllers)
- [Debugging Services](#debugging-services)
- [My service is missing endpoints](#my-service-is-missing-endpoints)
- [Network traffic is not forwarded](#network-traffic-is-not-forwarded)
- [More information](#more-information)
<!-- END MUNGE: GENERATED_TOC -->
@@ -46,46 +54,40 @@ your Service?
* [Debugging Services](#debugging-services)
### Debugging Pods
The first step in debugging a Pod is taking a look at it. For the purposes of example, imagine we have a pod
```my-pod``` which holds two containers ```container-1``` and ```container-2```
First, describe the pod. This will show the current state of the Pod and recent events.
The first step in debugging a Pod is taking a look at it. Check the current state of the Pod and recent events with the following command:
```sh
export POD_NAME=my-pod
kubectl describe pods ${POD_NAME}
```
Look at the state of the containers in the pod. Are they all ```Running```? Have there been recent restarts?
Depending on the state of the pod, you may want to:
* [Debug a pending pod](#debugging-pending-pods)
* [Debug a waiting pod](#debugging-waiting-pods)
* [Debug a crashing pod](#debugging-crashing-pods-or-otherwise-unhealthy-pods)
Continue debugging depending on the state of the pods.
#### Debuging Pending Pods
#### My pod stays pending
If a Pod is stuck in ```Pending``` it means that it can not be scheduled onto a node. Generally this is because
there are insufficient resources of one type or another that prevent scheduling. Look at the output of the
```kubectl describe ...``` command above. There should be messages from the scheduler about why it can not schedule
your pod. Reasons include:
You don't have enough resources. You may have exhausted the supply of CPU or Memory in your cluster, in this case
you need to delete Pods, adjust resource requests, or add new nodes to your cluster.
* **You don't have enough resources**: You may have exhausted the supply of CPU or Memory in your cluster, in this case
you need to delete Pods, adjust resource requests, or add new nodes to your cluster. See [Compute Resources document](compute-resources.md#my-pods-are-pending-with-event-message-failedscheduling) for more information.
You are using ```hostPort```. When you bind a Pod to a ```hostPort``` there are a limited number of places that pod can be
* **You are using ```hostPort```**: When you bind a Pod to a ```hostPort``` there are a limited number of places that pod can be
scheduled. In most cases, ```hostPort``` is unnecessary, try using a Service object to expose your Pod. If you do require
```hostPort``` then you can only schedule as many Pods as there are nodes in your Kubernetes cluster.
#### Debugging Waiting Pods
#### My pod stays waiting
If a Pod is stuck in the ```Waiting``` state, then it has been scheduled to a worker node, but it can't run on that machine.
Again, the information from ```kubectl describe ...``` should be informative. The most common cause of ```Waiting``` pods
is a failure to pull the image. Make sure that you have the name of the image correct. Have you pushed it to the repository?
Does it work if you run a manual ```docker pull <image>``` on your machine?
Again, the information from ```kubectl describe ...``` should be informative. The most common cause of ```Waiting``` pods is a failure to pull the image. There are three things to check:
* Make sure that you have the name of the image correct
* Have you pushed the image to the repository?
* Run a manual ```docker pull <image>``` on your machine to see if the image can be pulled.
#### Debugging Crashing or otherwise unhealthy pods
#### My pod is crashing or otherwise unhealthy
Let's suppose that ```container-2``` has been crash looping and you don't know why, you can take a look at the logs of
First, take a look at the logs of
the current container:
```sh
@@ -112,12 +114,12 @@ kubectl exec cassandra -- cat /var/log/cassandra/system.log
If none of these approaches work, you can find the host machine that the pod is running on and SSH into that host,
but this should generally not be necessary given tools in the Kubernetes API. Indeed if you find yourself needing to ssh into a machine, please file a
but this should generally not be necessary given tools in the Kubernetes API. Therefore, if you find yourself needing to ssh into a machine, please file a
feature request on GitHub describing your use case and why these tools are insufficient.
### Debugging Replication Controllers
Replication controllers are fairly straightforward. They can either create Pods or they can't. If they can't
create pods, then please refer to the [instructions above](#debugging-pods)
create pods, then please refer to the [instructions above](#debugging-pods) to debug your pods.
You can also use ```kubectl describe rc ${CONTROLLER_NAME}``` to introspect events related to the replication
controller.
@@ -126,8 +128,7 @@ controller.
Services provide load balancing across a set of pods. There are several common problems that can make Services
not work properly. The following instructions should help debug Service problems.
#### Verify that there are endpoints for the service
For every Service object, the apiserver makes an ```endpoints`` resource available.
First, verify that there are endpoints for the service. For every Service object, the apiserver makes an `endpoints` resource available.
You can view this resource with:
@@ -139,7 +140,7 @@ Make sure that the endpoints match up with the number of containers that you exp
For example, if your Service is for an nginx container with 3 replicas, you would expect to see three different
IP addresses in the Service's endpoints.
#### Missing endpoints
#### My service is missing endpoints
If you are missing endpoints, try listing pods using the labels that Service uses. Imagine that you have
a Service where the labels are:
```yaml
@@ -163,7 +164,7 @@ selected don't have that port listed, then they won't be added to the endpoints
Verify that the pod's ```containerPort``` matches up with the Service's ```containerPort```
#### Network traffic isn't forwarded
#### Network traffic is not forwarded
If you can connect to the service, but the connection is immediately dropped, and there are endpoints
in the endpoints list, it's likely that the proxy can't contact your pods.
@@ -173,6 +174,11 @@ check:
* Can you connect to your pods directly? Get the IP address for the Pod, and try to connect directly to that IP
* Is your application serving on the port that you configured? Kubernetes doesn't do port remapping, so if your application serves on 8080, the ```containerPort``` field needs to be 8080.
#### More information
If none of the above solves your problem, follow the instructions in [Debugging Service document](debugging-services.md) to make sure that your `Service` is running, has `Endpoints`, and your `Pods` are actually serving; you have DNS working, iptables rules installed, and kube-proxy does not seem to be misbehaving.
You may also visit [troubleshooting document](../troubleshooting.md) for more information.
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/user-guide/application-troubleshooting.md?pixel)]()