DevOps
Building a Custom CPU Autoscaler for Docker Swarm
July 202610 min read
Photo by M.Minderhoud at Dutch Wikipedia via Openverse (CC BY-SA 3.0)
No. Docker Swarm ships only the imperative docker service scale command with no controller that decides when to call it based on metrics. Kubernetes' HPA reads CPU, memory, or custom metrics and adjusts replica counts automatically on a control loop; on Swarm, that decision logic has to be built separately, which is exactly what this custom Bash daemon does.
Under-provisioning during a real spike degrades every in-flight request immediately, so the daemon should react fast: two consecutive samples above 70% CPU trigger a scale-out. Over-provisioning only wastes compute, so it is safe to be conservative and require three consecutive samples below 30% CPU before scaling in. Making both directions equally sensitive would either react too slowly to real load or scale in too eagerly on a temporary dip.
It prevents the daemon from reacting to the consequences of its own previous scaling action. Right after a scale-out, per-replica CPU naturally drops, which could satisfy a scale-in streak a few polls later even though real load hasn't receded. A 90-second cooldown blocks any further scaling decision after an action, regardless of streak state, until the new replica count has had time to reach a real steady state.
It was validated against a staged k6 load profile ramping from 1 to 5 to 20 virtual users over 9 minutes. The daemon scaled from one to two replicas at +52 seconds from ramp onset, inside a 60-second target, and per-replica CPU settled to the predicted roughly 60% range. A spurious 92% CPU spike that arrived during the post-scale-out cooldown was correctly suppressed, confirming the anti-flapping mechanism worked as designed.
No. In the broader thesis work behind this daemon, the platform's real throughput ceiling turned out to be a database-level row lock on a shared configuration table that no amount of replica scaling could remove, because every request updated the same row. A CPU-based autoscaler resolves CPU saturation but cannot see past the layer it measures; bottlenecks at the database or application-transaction level need a different fix entirely.
Photo by M.Minderhoud at Dutch Wikipedia via Openverse (CC BY-SA 3.0)
Key Takeaway
Docker Swarm has no built-in autoscaler like Kubernetes HPA, so this post builds one from scratch: an under-80-line Bash daemon that polls CPU every 10 seconds, needs two consecutive readings above 70% to scale out but three below 30% to scale in, and enforces a 90-second cooldown after every action to stop flapping.
During my undergraduate thesis at Swiss German University, done in partnership with Commsult Indonesia, I benchmarked a Java Spring Boot logistics platform called Ontego Traces that runs entirely on Docker Swarm. One gap became obvious in the first week of design work: Docker Swarm has no built-in equivalent of Kubernetes' Horizontal Pod Autoscaler. There is no controller that watches a metric and adjusts replica count for you. If a service under Swarm needs to scale in response to load, someone has to build that logic by hand. So I did. This post walks through the design of that autoscaler, why I chose asymmetric thresholds instead of a single symmetric one, and the actual numbers from a staged load test where it caught a real scaling event at 52 seconds and correctly ignored a fake one.
Kubernetes ships a Horizontal Pod Autoscaler that reads metrics from the metrics-server or a custom adapter and adjusts a Deployment's replica count on a control loop, typically every 15 seconds. Docker Swarm was designed for a different niche: single-binary simplicity, built into the Docker Engine itself, with no separate control plane to operate. That simplicity is exactly why teams choose Swarm for smaller platforms, but it means Swarm ships only a static primitive, docker service scale, and nothing that decides when to call it. For Ontego Traces, every service ran as a single replica with no scaling logic at all before this work. That was fine at low traffic, but the platform's own health-check history showed response time spikes above 18 seconds during uncontrolled load, which is exactly the failure mode an autoscaler exists to prevent.
The autoscaler itself is packaged as a Docker Swarm service running on the manager node, with the Docker socket mounted so it can call docker stats and docker service scale directly. It polls the CPU usage of every running task belonging to the target service every 10 seconds, averages that reading, and feeds it into a decision loop. The polling interval had to balance two costs: too frequent, and the daemon adds noticeable overhead itself; too infrequent, and the reaction time to a real spike misses the target window. Ten seconds landed in the middle and matched the granularity docker stats naturally reports at.
The core design decision is that scaling out and scaling in are not symmetric problems. Under-provisioning during a traffic spike degrades every request in flight, so the daemon should react as soon as it has reasonable confidence there is a real spike, not a single noisy reading. Over-provisioning, on the other hand, wastes compute but does not break anything, so the daemon can afford to be conservative and wait for a sustained trend before scaling in. I encoded that asymmetry directly into the streak counters: two consecutive 10-second samples above 70% CPU trigger a scale-out, but three consecutive samples below 30% CPU are required before a scale-in fires. A single high reading, which could be a garbage collection pause or a transient burst, is not enough to trigger anything on its own.
Streak consensus alone does not fully prevent oscillation, because a service that just scaled out will usually see its per-replica CPU drop immediately, which could satisfy a scale-in streak a few polls later even though the underlying load has not actually receded. To stop that, every scaling action, in either direction, starts a 90-second cooldown during which no further scaling decision is executed regardless of what the streak counters say. Ninety seconds was chosen because it comfortably exceeds the time it takes a newly scheduled Swarm task to pass its health check and start absorbing real traffic, so the daemon always gets a fair read of the new steady state before it is allowed to act again.
If you are adapting this pattern, resist the temptation to make the cooldown symmetric with the streak requirement. The cooldown protects against the daemon reacting to its own previous action; the streak count protects against reacting to noise. They solve different problems and both matter.
The full daemon is under 80 lines of Bash. It runs docker stats with the --no-stream flag so each call returns a single snapshot instead of a live stream, averages the CPU percentage across every task container belonging to the service, then runs that average through the streak-and-cooldown logic before calling docker service scale only when a decision actually fires. The snippet below is the real core of the loop, trimmed of logging and edge-case handling for readability.
#!/usr/bin/env bash
# autoscaler.sh — polls a Swarm service's CPU and scales with
# asymmetric streak consensus + cooldown. Runs as a Swarm service
# on the manager node (needs the manager socket mounted read-only).
SERVICE="tdp_tour-data-provider"
SCALE_OUT_THRESHOLD=70 # % CPU
SCALE_IN_THRESHOLD=30 # % CPU
SCALE_OUT_STREAK_NEEDED=2 # fast to react
SCALE_IN_STREAK_NEEDED=3 # slow to react (avoid flapping)
COOLDOWN_SECONDS=90
POLL_INTERVAL=10
MAX_REPLICAS=4
MIN_REPLICAS=1
out_streak=0
in_streak=0
last_scale_ts=0
while true; do
now=$(date +%s)
# Average CPU % across every running task of the service, this poll
cpu=$(docker stats --no-stream --format '{{.CPUPerc}}' \
$(docker ps --filter "label=com.docker.swarm.service.name=${SERVICE}" -q) \
| tr -d '%' | awk '{sum+=$1; n++} END {if (n>0) print sum/n; else print 0}')
cooling_down=$(( now - last_scale_ts < COOLDOWN_SECONDS ))
if (( $(echo "$cpu > $SCALE_OUT_THRESHOLD" | bc -l) )); then
out_streak=$((out_streak + 1))
in_streak=0
elif (( $(echo "$cpu < $SCALE_IN_THRESHOLD" | bc -l) )); then
in_streak=$((in_streak + 1))
out_streak=0
else
out_streak=0
in_streak=0
fi
current=$(docker service inspect "$SERVICE" \
--format '{{.Spec.Mode.Replicated.Replicas}}')
if (( out_streak >= SCALE_OUT_STREAK_NEEDED && !cooling_down \
&& current < MAX_REPLICAS )); then
new=$((current + 1))
echo "[scale-out] cpu=${cpu}% streak=${out_streak} ${current}->${new}"
docker service scale "${SERVICE}=${new}"
last_scale_ts=$now
out_streak=0
elif (( in_streak >= SCALE_IN_STREAK_NEEDED && !cooling_down \
&& current > MIN_REPLICAS )); then
new=$((current - 1))
echo "[scale-in] cpu=${cpu}% streak=${in_streak} ${current}->${new}"
docker service scale "${SERVICE}=${new}"
last_scale_ts=$now
in_streak=0
fi
sleep "$POLL_INTERVAL"
done
One detail worth calling out: docker stats --no-stream still needs a live list of container IDs on every poll, because Swarm reschedules tasks and container IDs are not stable across a scaling event or a node failure. The daemon re-resolves the container ID list from docker ps with a Swarm service-name label filter on every iteration, rather than caching it, which costs a small amount of CPU on the manager node but avoids querying stats for a container that no longer exists.
docker stats reports CPU as a percentage of a single core by default, so on a multi-core host a container can legitimately report over 100%. Average the readings across containers, not against the host's total core count, or your threshold comparisons will be silently wrong the moment a container spreads work across more than one core.
To validate the daemon end to end rather than just unit-test its logic, I ran it against a staged k6 load profile that ramped from 1 to 5 to 20 virtual users over 9 minutes, starting from a single replica of the Tour Data Provider service. This mirrors a realistic traffic ramp rather than an instant step, which is the harder case for any autoscaler because the signal builds gradually instead of announcing itself immediately.
Single-replica CPU reached 97% and then 79% on two consecutive 10-second samples, satisfying the scale-out streak. The daemon scaled the service from one replica to two at +52 seconds from the start of the ramp, inside the 60-second target I had set for the design. Per-replica CPU then settled to 62% and 59%, matching the roughly 60% per-replica load I had predicted for two replicas splitting the same traffic. When the ramp receded at the end of the profile, three consecutive samples at 31%, 23%, and 15% correctly triggered a scale-in back to one replica.
The result I actually care most about: during the 90-second cooldown that followed the scale-out, a spurious 92% CPU spike arrived and was correctly suppressed. That single suppressed event is the entire point of the cooldown window. Without it, that spike would have triggered a second scale-out purely on the strength of a transient reading, which is exactly the flapping behavior the asymmetric design exists to prevent.
The autoscaler resolves CPU saturation reliably and within its target reaction window, but it is worth being honest about its limits. In the broader thesis work, the same platform's throughput ceiling turned out to be a database-level row lock contention issue that no amount of replica scaling could fix, because every request updated the same tenant configuration row. The autoscaler converges to that same ceiling once CPU stops being the binding constraint; it cannot see past the layer it measures. A CPU-only autoscaler is also a known simplification versus bi-metric approaches that factor in memory or request latency, which I call out explicitly as a direction for follow-up work rather than a solved problem.
If you are building something similar for your own Swarm deployment, log every streak transition and every cooldown-suppressed decision to a file the daemon can hand off to your observability stack. Being able to show, after the fact, exactly which spike was ignored and why is what makes an anti-flapping mechanism trustworthy to the rest of the team.
If you are running production workloads on Docker Swarm rather than Kubernetes, autoscaling is not a checkbox you flip on, it is a small piece of infrastructure you have to design and own. The core ideas here, asymmetric thresholds, streak-based consensus instead of single-sample triggers, and an explicit cooldown, are not specific to Docker or to this platform. They apply to any home-grown scaling controller you build against any orchestrator's imperative scale API. The trade-off is honest: you get exactly the scaling behavior you designed, at the cost of building and testing it yourself.