For the love of god, stop using CPU limits on Kubernetes (updated)

Many people think you need CPU limits on Kubernetes but this isn't true. In most cases, Kubernetes CPU limits do more harm than help.

I will explain why CPU limits are harmful with three analogies between CPU starved pods and thirsty explorers lost in a desert. We will call our intrepid explorers Marcus and Teresa.

In our stories, CPU will be water and CPU starvation will be death. Like CPU, water in our story will be a renewable resource. In simple terms, if you have 100% CPU usage at a given minute, that doesn't "use up" the CPU for the next minute. CPU renews itself from moment to moment.

Three colorful analogies about Kubernetes CPU Limits

Marcus and Teresa are travelling in the desert. They have a magical water bottle that produces 3 liters a day. Each person needs 1 liter a day to survive.

Story 1 - without limits, without requests: Marcus is greedy so he drinks all the water before Teresa can drink any. Teresa dies of thirst. This is because there were no limits or requests. Markus was able to drink all the water, cause CPU starvation, and Teresa was CPU throttled.

Story 2 - with limits, with or without requests: Teresa gets very ill one day and needs some extra water. Markus drinks his one liter and there are two liters left over. Teresa drinks one liter and now there is one liter remaining. Markus wont let Teresa drink it because her limit is 1 liter per day so she dies of thirst. This is what happens when you have CPU limits. Resources are available but you aren't allowed to use them.

Story 3 - without limits, with requests: Marcus gets very ill and needs extra water one day. He tries to drink the entire bottle but is stopped when only 1 liter remains in the bottle. This is saved for Teresa because she needs 1 liter a day. She drinks her 1 liter. Nothing remains. They both live. This is what happens when you have no CPU limits but you do have requests. All is good.

The above stories are surprisingly precise analogies for why CPU limits are considered harmful.

Don't like analogies? I wrote a more technical explanation of this when explaining the Prometheus CPU throttling alert, CPUThrottlingHigh.

Preventing CPU throttling and insufficient CPU without limits

Many people think you need limits to prevent one pod from interfering with another pod. This is not true! You can remove Kubernetes CPU limits and still prevent a CPU hungry pod from causing CPU starvation! The trick is to just define CPU requests.

Here is a snippet from the original Kubernetes documentation:

The important part is highlighted. Pods always get the CPU requested by their CPU request! (They can take advantage of excess CPU too if they have no limit.)

Disable your CPU limits! If you give all your K8s pods accurate CPU requests, then no-one can throttle them because CPU is reserved for them if they need it. This has nothing to do with limits.

Kubernetes Requests vs Limits

Lets summarize the difference between CPU limits and requests in a table:

I'm assuming for simplicity that either all pods have CPU limits or none have them. The same is true for CPU requests. This makes everything simpler to reason about.

More common misunderstandings about Kubernetes resource management

I've started a video series to elaborate on the concepts in this post, as well as other misconceptions. Check it out.

Some content could not be imported from the original document. View content ↗

Best practices for CPU limits and requests on Kubernetes

Lets summarize:

Use CPU requests for everything
Make sure they are accurate
Do not use CPU limits.

Tim Hockin (one of the original Kubernetes maintainers at Google) has recommended the same for years.

What about memory limits and requests?

Everything in this post is about CPU and not memory. Memory is different because it is non-compressible - once you give memory you can't take it away without killing the process. We've covered the best practices for Kubernetes memory limits here. In short, our bottom line recommendation is:

Always use memory limits
Always use memory requests
Always set your memory requests equal to your limitsS

Other reasons why CPU throttling can occur on Kubernetes

This post focused on K8s CPU limits, but there are actually other reasons why CPU throttling can happen. This is a good opportunity to plug Robusta.dev, which tells you the root cause of Prometheus alerts on Kubernetes. It analyzes your alerts and adds missing context. This is true for all alerts, including Kubernetes high CPU throttling alerts.