# Kubernetes HPA Autoscaling: Surviving a PaaS Traffic Spike

> Autoscaling was on, and the opening seconds of the spike were still served by the pods I already had. The real Kubernetes HPA timeline on a managed PaaS, default by default.

URL: https://www.matthewswong.com/en/blog/helipod-hpa-autoscaling-traffic-spike/

- Author: Matthews Wong (https://www.matthewswong.com/en/about/)
- Published: 2026-09-03
- Updated: 2026-09-03
- Category: Cloud

> **Key takeaway:** Kubernetes HPA autoscaling on a managed PaaS does not serve the start of a traffic spike. The controller loop runs every fifteen seconds by default, a scale-up still waits for pod scheduling and a readiness probe, and CPU utilisation is measured against resources.requests, so the pods already running absorb the opening burst.

Helipod's autoscaling panel is three fields: a minimum replica count, a maximum, and a CPU target. The screenshot on its own front page shows min 1, max 10 and a target of 70 per cent, and the promise beside it is accurate as far as it goes — you set the limit, and HPA handles the rest. What those three fields hide is that everything happening between the moment traffic arrives and the moment a new pod serves its first request is decided by Kubernetes defaults nobody in that panel ever chose.

So I read the Kubernetes Horizontal Pod Autoscaler documentation with a stopwatch in mind, and wrote down every default that fires during a spike: the sync period, the scale-up policies, the tolerance, the two readiness delays. This post is that timeline, and then the four application-level facts no platform can fix for you, however good its dashboard is.

## Autoscaling on is not the same as the spike served

No autoscaler adds capacity before the load that justifies it exists. The Horizontal Pod Autoscaler is a control loop: it reads a metric, compares it to a target, and asks a Deployment for a different replica count. Every one of those steps happens after your traffic has already arrived and already been served — or already been queued — by the pods you had. There is no predictive path in the CPU-based configuration a PaaS exposes.

The Kubernetes documentation says this plainly: horizontal pod autoscaling is implemented as a control loop that runs intermittently, and it is not a continuous process. Treating it as insurance against a traffic spike is a category error. It is insurance against a spike that lasts longer than a couple of minutes. The opening burst is a capacity-planning problem, and the only setting that touches it is the minimum replica count you chose before anything happened.

## Three fields in the dashboard, a dozen defaults underneath

Those three fields map onto a HorizontalPodAutoscaler object, and that object has a behavior section the dashboard never shows you. Kubernetes documents its defaults explicitly, which means you can write out exactly what you agreed to when you flipped the switch.

The asymmetry is the part worth memorising. Scaling up has a stabilization window of zero seconds, so the controller acts on the first reading that crosses the target. Scaling down has a window of 300 seconds and takes the highest recommendation from anywhere inside it — a rolling maximum, designed to stop the controller removing a pod it will need again a moment later. The platform is biased towards keeping capacity, which is the right bias, and it also means an over-scaled app stays over-scaled for five minutes after the spike ends. On per-resource daily billing that is a line item, not a rounding error.

## The formula, run against a spike

The controller's entire decision is one ratio: desiredReplicas is currentReplicas multiplied by the current metric value over the desired metric value, rounded up. It skips the change altogether when the ratio is close enough to 1.0, inside a tolerance that defaults to 0.1. With a 70 per cent CPU target, that means anything between roughly 63 and 77 per cent average utilisation is treated as steady state and ignored.

Two things in that arithmetic surprised me. First, the metric rather than the policy is usually what limits the ramp: the default scale-up policies would allow adding four pods or doubling every fifteen seconds, but a pod whose CPU limit equals its CPU request can never report more than 100 per cent utilisation, so the ratio is capped and the controller asks for less than it is permitted to give. Second, the dampening. When some pods are not yet Ready, the controller recalculates on the assumption that they consume nothing, specifically to avoid overshooting — so the moment you most want it to be aggressive is the moment it is deliberately conservative.

## The clock runs in fifteen-second ticks

The loop's interval is set by the sync-period flag on kube-controller-manager, and it defaults to fifteen seconds. On a managed platform that is a cluster-wide setting, which means it is not yours. A spike that begins one second after a tick waits almost a full period simply to be noticed, and nothing about your app or your configuration changes that.

After the decision, the replica count on the Deployment changes immediately and the useful work has not started. The scheduler still has to find a node with room for the pod's CPU request, the container image still has to be present or pulled, the process still has to boot, and the readiness probe still has to pass before the Service will route a single request to it. None of those four steps has a documented time budget, because all four depend on your image, your node pool and your application. They are also the only part of the timeline you can actually shorten.

*Only the bottom band is something a customer can feel; every band above it is platform machinery that reports itself as healthy while the queue on your original replicas keeps growing. Notice that the two bands with real numbers are the two you cannot configure from a PaaS dashboard.*

> The one setting that changes what happens in the first fifteen seconds is minReplicas. Set it from the opening burst you expect rather than from your average load, accept that you are paying for idle pods between spikes, and treat that as the price of the window no autoscaler can cover. Every other knob only affects what happens after the damage.

## resources.requests is the denominator of every decision

A CPU target is not a percentage of the node or of your plan. The controller calculates utilisation as a percentage of the equivalent resource request on the containers in the pod, so resources.requests.cpu is the denominator of every scaling decision the platform will ever make about your app. Kubernetes defines one CPU unit as one physical core and 100m as one hundred millicpu, so these are absolute quantities: a wrong one does not shift the target slightly, it changes what the target means.

The failure I would not have predicted is the silent one. A request far larger than the app can consume makes the target unreachable, so the HPA reports comfortable utilisation while requests are timing out, and every dashboard stays green. A missing request is worse: the documentation states that if a container has no relevant resource request set, CPU utilisation for the pod is undefined and the autoscaler takes no action for that metric at all. Autoscaling is on, configured, displayed, and doing nothing.

> Setting requests equal to limits is the safe default for scheduling and the quiet enemy of fast scaling. CPU limits are enforced by throttling, so a pod pinned at its limit can never report more than 100 per cent utilisation, and against a 70 per cent target the ratio cannot pass about 1.43 — roughly one extra pod per existing pod, per tick. Leave headroom between request and limit if you want the controller to be able to ask for a big jump.

## A new pod is not a warm pod

Readiness, not existence, is what puts a pod into service. If a readiness probe returns a failed state, the EndpointSlice controller removes the pod's IP address from the EndpointSlices of every Service that matches it, and the probe keeps running for the container's whole lifecycle rather than only at startup. The probe is therefore the admission gate for all your new capacity, and how honest it is decides whether the spike is served or merely counted.

The HPA has its own opinion about brand new pods, and it is a conservative one. A pod that is unready shortly after starting is treated as still initialising for an initial-readiness-delay that defaults to thirty seconds, and its CPU is ignored for a cpu-initialization-period that defaults to five minutes unless the sample was taken entirely while it was Ready. On top of that, a fresh Next.js pod is cold in a way Kubernetes cannot see: the Next.js self-hosting guide notes that generated cache assets are stored in memory and on disk, and that under a container orchestration platform like Kubernetes each pod holds its own copy of the cache. Scaling from one pod to ten multiplies your origin work by ten until those caches fill.

> Helipod's real-time log streaming shows every pod at once in the browser, and its browser terminal opens a shell into a running production pod without SSH keys or a VPN. That pairing is what turns this timeline from theory into something observable: you see the new pod's first log line, its first failed readiness check and its first slow query, in the order they actually happened.

## A CPU target is blind to a connection pool

The bottleneck a CPU-based autoscaler cannot see is the one where your app is waiting rather than working. A pod blocked on a database connection burns almost no CPU, so utilisation falls exactly when the user experience gets worse, and the honest reading of that metric is that the app needs fewer pods. Meanwhile the pool configuration you wrote when the app was a single pod is now multiplied by the replica count.

| What you see during the spike | What the CPU target sees | What actually helps |
| --- | --- | --- |
| Requests queue while CPU sits near the target | A ratio inside the 0.1 tolerance, so no action at all | The limit is concurrency, not CPU — scale on queue depth or in-flight requests instead |
| Latency climbs while CPU falls | A candidate for scaling down | Look at I/O: a pod waiting on a database burns no CPU while it waits |
| Every new pod errors on its first query | Unready pods, assumed to consume nothing, so the scale-up is dampened | Divide the connection pool by maxReplicas before the spike, not after it |
| The first requests to each new pod are slow | Nothing — a cache miss is not a metric it collects | A shared cache handler, or an explicit decision to accept a cold tail on every scale-up |
| Replicas stay at the ceiling long after traffic drops | The highest recommendation from the last 300 seconds | Nothing, and that is correct — shorten the stabilization window only if you can prove the spike is over |

Every row has the same shape: the metric is behaving correctly and answering a different question from the one you are asking. That is not an argument against CPU-based autoscaling, which is cheap, needs no extra components and handles the common case well. It is an argument for knowing which of your failures it can see.

## What I check before a launch now

None of this needed a different platform. It needed five decisions taken before the traffic arrived rather than during it.

1. Measure the app's real CPU under load and set resources.requests.cpu to that figure, because it is the denominator of every scaling decision that follows.
2. Set minReplicas from the opening burst rather than the average, since nothing else affects the first fifteen seconds.
3. Point the readiness probe at an endpoint that fails until the app can genuinely serve a request, and add a startup probe instead of an initialDelaySeconds if warm-up is expensive.
4. Divide the database connection pool by maxReplicas, so the ceiling of the autoscaler is also the ceiling of the connection count.
5. Watch the logs of every pod at once during the first real spike, so the timeline is observed rather than assumed.

Autoscaling stopped being a switch for me and became four numbers I have to be able to defend: the request, the target, the minimum and the maximum. A managed PaaS runs the loop reliably, and that is genuinely most of the work. Deciding what the loop measures, and having enough capacity already running to survive the fifteen seconds before it notices anything, is still mine.

## Sources

- Kubernetes documentation — Horizontal Pod Autoscaling: the control loop, the sync period, the scaling formula, the tolerance and the behavior defaults
- Kubernetes documentation — Resource Management for Pods and Containers: CPU units, requests, and limits enforced by throttling
- Kubernetes documentation — Pod Lifecycle: container probes, and how readiness controls a Service's endpoints
- Next.js documentation — Self-Hosting: caching and ISR, and why each pod holds its own copy of the cache
- Helipod — the Indonesian PaaS whose autoscaling panel exposes a minimum, a maximum and a CPU target

*In the left and right columns CPU stays low while latency climbs, so the autoscaler correctly decides to do nothing about a real outage.*

## Source links

- https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/
- https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
- https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/
- https://nextjs.org/docs/app/guides/self-hosting
- https://helipod.io/en

## FAQ

### Does turning on autoscaling protect my app from a traffic spike?

Not the beginning of it. The Kubernetes Horizontal Pod Autoscaler is a control loop that reacts to a metric it has already measured, so the opening seconds of any spike are served by the replicas that were already running. Autoscaling protects you from a spike that lasts longer than a couple of minutes; minReplicas is what protects you from the first one.

### How quickly does Kubernetes HPA add a pod during a traffic spike?

The control loop runs intermittently, at an interval that defaults to fifteen seconds, so a spike can wait almost a full period simply to be noticed. Scale-up has no stabilization window by default, so the decision itself is immediate once the metric is read. What follows has no documented budget: scheduling, image pull, process start and the readiness probe all depend on your image and your cluster.

### Why is my HPA not scaling even though the app is clearly overloaded?

The usual cause is resources.requests. Utilisation is calculated as a percentage of the container's CPU request, so a request the app can never saturate keeps the measured percentage low forever. If no CPU request is set at all, the Kubernetes documentation states that CPU utilisation for the pod is undefined and the autoscaler takes no action for that metric. The other common cause is an I/O bottleneck, where the app waits instead of burning CPU.

### What should I set resources.requests.cpu to when scaling on a CPU target?

Set it to the CPU one pod actually uses under load, measured rather than guessed, because it is the denominator of every scaling decision. Keep the limit above the request so measured utilisation can exceed 100 per cent. When limit equals request, throttling caps utilisation at 100 per cent, and against a 70 per cent target that caps the scale-up ratio at roughly 1.43 per tick.

### Why do my replicas stay high long after the traffic spike ended?

That is the scale-down stabilization window, which defaults to 300 seconds. The controller takes the highest recommendation from anywhere inside that window, a rolling maximum that stops it removing a pod it will need again moments later. It is deliberate behaviour, and shortening it trades cost against the risk of thrashing during a spike that is not actually over.

---

- HTML version: https://www.matthewswong.com/en/blog/helipod-hpa-autoscaling-traffic-spike/
- Bahasa Indonesia: https://www.matthewswong.com/id/blog/helipod-hpa-autoscaling-traffic-spike/
- Site index for agents: https://www.matthewswong.com/llms.txt
- Sitemap: https://www.matthewswong.com/sitemap.xml
- API description (OpenAPI): https://www.matthewswong.com/openapi.json