Resolving resource limit issues in Cloud Service Mesh

This section explains common Cloud Service Mesh problems and how to resolvethem. If you need additional assistance, seeGetting support.

Cloud Service Mesh resource limit problems can be caused by any of thefollowing:

• LimitRange objects created in theistio-system namespace or any namespacewith automatic sidecar injection enabled.
• User-defined limits that are set too low.
• Nodes run out of memory or other resources.

Potential symptoms of resource problems:

• Cloud Service Mesh repeatedly not receiving configuration from the control planeindicated by the error,Envoy proxy NOT ready. Seeing this error a few timesat startup is normal, but otherwise it is a concern.
• Networking problems with some pods or nodes that become unreachable.
• istioctl proxy-status showingSTALE statuses in the output.
• OOMKilled messages in the logs of a node.
• Memory usage by containers:kubectl top pod POD_NAME --containers.
• Memory usage by pods inside a node:kubectl top node my-node.
• Envoy out of memory:kubectl get pods shows statusOOMKilled in the output.

Sidecars take a long time to receive configuration

Slow configuration propagation can occur due to insufficient resources allocatedtoistiod or an excessively large cluster size.

There are several possible solutions to this problem:

1. For in-cluster Cloud Service Mesh, if your monitoring tools (prometheus,stackdriver, etc.) show high utilization of a resource byistiod, increasethe allocation of that resource, for example increase the CPU or memory limitof theistiod deployment. This is a temporary solution and we recommendedthat you investigate methods for reducing resource consumption.

2. If you encounter this issue in a large cluster or deployment, reduce theamount of configuration state pushed to each proxy by configuringSidecar resources.

3. For in-cluster Cloud Service Mesh, if the problem persists, tryhorizontally scalingistiod.

4. If all other troubleshooting steps fail to resolve the problem, report a bugdetailing your deployment and the observed problems. Followthese stepsto include a CPU/Memory profile in the bug report if possible, along with adetailed description of cluster size, number of pods, and number of services.