System design, step by step · Lesson 2 of 7 · 13 min

How to Scale a Web Application: Stateless Servers to CDN

How to scale a web application step by step: stateless copies behind a load balancer, sessions in a shared store, then a CDN and a cache to absorb reads.

By Matthews WongReviewed on 13 October 2026

Key takeaway

Scaling a web application means adding capacity where it is cheapest first: make the app servers stateless so identical copies can run behind a load balancer, move sessions and other state into a shared store, then let a CDN and a cache answer repeated reads, so the origin and the database see only a fraction of the traffic.

You will be able to

Explain what makes a server stateless, and why sticky sessions only hide the problem.
Trace a request through a load balancer, a session store, a CDN and a cache, and say which layer answers it.
Decide what a shared cache may store, and use versioned URLs so nothing needs purging.

Watch it work

Step through the diagram

Press Play, or step with the arrows. Every step is written out below the diagram.

One server to a stateless tier, step by step

A second app copy behind nginx loses sessions kept in memory; moving them to Redis makes every copy stateless, a CDN answers the repeated public requests, Redis caches hot reads, and private pages always reach the origin.

Today one app server does everything: it renders pages, keeps each login in its own memory and reads the database. Dashed boxes are what scaling will add.

Keys: ← → step · Home / End · K play or pause

Today one app server does everything: it renders pages, keeps each login in its own memory and reads the database. Dashed boxes are what scaling will add.
Traffic grows, so a second copy joins behind nginx, which now deals requests in turn. Ana logs in on app 1, and app 1 keeps her session in memory.
Predict the next step Ana's next request goes to app 2. What does app 2 see? (Ana, still logged in · A stranger with no session)
Round robin sends her next request to app 2, which has never seen her session. To app 2 she is a stranger: logged out, cart empty.
Move sessions to a shared store with an expiry, such as Redis. Any copy can now load Ana's session by id, so any copy can serve any request.
SET session:ana '...' EX 1800
Most requests are identical for every visitor: images, scripts, public pages. A CDN edge near Ana keeps copies and answers repeats itself; the origin never sees them.
Cache-Control: public, max-age=31536000, immutable
Reads that reach the app repeat too. Redis can keep hot results beside the sessions, so the database serves only cache misses and writes.
GET post:42Predict the next step Ana opens her own cart page. Who should answer it? (The CDN's shared copy · The origin app, every time)
Her cart holds her data, so no shared cache may store it. The response says private, no-store, and the CDN sends every cart request to the origin.
Cache-Control: private, no-store

What makes a web server stateless, and why does scaling need it?

There are two ways to add capacity. Scaling up gives one machine more CPU and memory, and your code never notices. Scaling out adds more machines, and suddenly every request may land on a different process. Start with up because it needs no code change, and move out when you must survive a machine failure or the biggest machine is no longer enough.

How does a load balancer spread requests, and why not sticky sessions?

A load balancer sits in front of the copies and picks one for each request. nginx defaults to round robin, dealing requests in turn, and it runs passive health checks: a server whose response fails is marked failed and avoided for a while, set by max_fails and fail_timeout. Spreading requests across copies raises throughput and lets the tier survive one copy dying.

What should a CDN cache, and what must it never store?

A CDN is a geographically distributed network of proxy servers that keeps copies of your responses close to visitors. Each edge is a shared cache, one that stores responses for more than one user, so its value is the hit ratio: every request it answers is one the origin never sees. Cache-Control tells it what to do. s-maxage sets the lifetime for shared caches only, so public HTML can send max-age=0 with a short s-maxage: the edge absorbs the traffic while browsers still revalidate.

Where does an application cache fit once the CDN is in place?

Requests that pass the edge still repeat: the same product, the same post, the same profile. Cache-aside handles them in the app: look in the cache first; on a miss, read the database, store the result with an expiry and return it. On a write, update the database first and then delete the cached entry, so the next read loads the fresh row.

The real config behind this lesson

Excerpts from the repository that builds and deploys this website, quoted as they are.

lib/rate-limit.ts:1-11

/**
 * Simple in-memory rate limiter for API routes
 * In production, consider using Redis (Upstash) for distributed rate limiting
 */

interface RateLimitEntry {
  count: number;
  resetTime: number;
}

const rateLimitMap = new Map<string, RateLimitEntry>();

This site's API rate limiter keeps its counts in a Map inside the Node process. With one container, as the site runs today, that is correct. Run two copies behind a load balancer and each counts on its own, so a client could get up to twice the limit, and every deploy, which replaces the container, resets the counts. The file's own comment names the fix: a shared store such as Redis.

next.config.ts:198-211

        // The films, their posters and captions. A re-render keeps the file
        // names, so only a request carrying the render stamp
        // (lib/film-manifest.ts prints `?v=` on every URL) may be cached as
        // immutable; Cloudflare then serves the films from its edge. A bare
        // URL, from an old link, keeps revalidating against the origin.
        source: "/videos/:path*",
        has: [{ type: "query", key: "v" }],
        headers: [
          {
            key: "Cache-Control",
            value: "public, max-age=31536000, immutable",
          },
        ],

Cache busting in this site's own config. A film keeps its file name when it is re-rendered, so only a URL carrying the render stamp, ?v=, is marked immutable for a year, and Cloudflare serves those from its edge. A bare URL from an old link keeps revalidating against the origin, so nobody is stuck with an outdated film.

What breaks

Mistakes that cost real time

Sticky sessions as the fix
Pinning each user to one copy hides in-memory sessions instead of removing them. When that copy restarts, is removed or fails, everyone pinned to it is logged out. Move sessions to a shared store and drop the stickiness.
State you forgot was state
Sessions are the obvious case. Counters such as a rate limiter, uploads written to local disk and timers inside the app break the same way: each copy counts alone, a file exists on one disk only, and a scheduled job runs once per copy.
A personalised page in a shared cache
Cache a page that greets one user and the next visitor gets their name, or worse, their invoice. Vary: Cookie does not fix it; it stores one copy per cookie and still keeps private data on a shared system. Mark it private or no-store.
What I'd change here: immutable on a name that can change
This site sends public, max-age=31536000, immutable for everything under /images/, yet blog covers are re-rendered under the same file name when the cover template changes. A browser that already holds an old cover may keep showing it for up to a year. Versioning the cover URL, as the films do with ?v=, would fix it.

Scale up or scale out
	Scale up (bigger server)	Scale out (more copies)
Code changes	None: the app does not notice	The app must be stateless
If the machine dies	All capacity is gone at once	One share of capacity is gone
Ceiling	The largest machine you can rent	Much higher, bounded by shared parts like the database
What it needs	A restart or migration to resize	A load balancer and a shared store

Try it yourself

Do it on your own machine

You need Node and Docker. Run two copies of a tiny server that counts visits in memory, put nginx in front of them, watch the count split, then move it to Redis.

Save server.mjs. It counts visits in its own memory, which is exactly the state that breaks scaling out.

// server.mjs: counts visits in this process's memory
import http from "node:http";
const port = Number(process.env.PORT);
let visits = 0;
http
  .createServer((req, res) => {
    visits += 1;
    res.end(`port ${port}: visit ${visits}\n`);
  })
  .listen(port);

Start two copies on different ports.

PORT=3001 node server.mjs &
PORT=3002 node server.mjs &

Save nginx.conf and run nginx in Docker as a round-robin load balancer on port 8080. The --add-host flag lets the container reach your machine on Linux too.

events {}
http {
  upstream app {
    server host.docker.internal:3001;
    server host.docker.internal:3002;
  }
  server {
    listen 8080;
    location / { proxy_pass http://app; }
  }
}

# then:
docker run --rm -p 8080:8080 --add-host=host.docker.internal:host-gateway \
  -v "$PWD/nginx.conf:/etc/nginx/nginx.conf:ro" nginx:alpine

4
In another terminal, send four requests. Each copy keeps its own count, so visit 1 appears twice: the state is split across the copies.
```
for i in 1 2 3 4; do curl -s localhost:8080; done
```

Stop both copies, start Redis, install the client and replace the counter with a shared one. Start both copies again: the count now rises across them, 1, 2, 3, 4.

docker run --rm -d -p 6379:6379 redis:7-alpine
npm install redis

// server.mjs: the count now lives in Redis, shared by every copy
import http from "node:http";
import { createClient } from "redis";
const redis = await createClient().connect();
const port = Number(process.env.PORT);
http
  .createServer(async (req, res) => {
    const visits = await redis.incr("visits");
    res.end(`port ${port}: visit ${visits}\n`);
  })
  .listen(port);

Quick quiz

Three questions before you move on

Pick an answer to see why it is right or wrong. Nothing is saved or graded, so guess freely.

1.You run three copies behind nginx with round robin, and users say they get logged out at random. What is the most likely cause?
2.Which response is safe to keep at a shared CDN edge for a long time?
3.A Redis cache in front of the database has an 80 percent hit ratio, and 1,000 reads per second reach the app. How many reach the database?

Think it through

Answer in your head, then check

1.Name two kinds of state, besides sessions, that stop an app from scaling out.Show the answer
Answer:In-memory counters such as a rate limiter, and files written to local disk. Timers inside the app are a third: with three copies they fire three times.
2.Why send max-age=0 together with a positive s-maxage on public HTML?Show the answer
Answer:s-maxage applies only to shared caches, so the CDN edge keeps and serves the page while each browser revalidates, and a new deploy reaches readers quickly.
3.Why is a versioned URL better than purging the CDN?Show the answer
Answer:Nothing has to be deleted. New content arrives at a new URL, so every cache misses once and fetches it, while the old URL can stay cached for as long as it likes.

Read the module

Every post in this module, in order

The lesson above is the map. Each post goes deep on one part of it, with the full worked examples.

Sources

Primary sources checked on the research date. Follow them for the current version.

Previous lessonSystem Design Estimation for Beginners: Users to Servers Next lessonHow to Scale a Database: Indexes, Replicas and Sharding