CDN: How Content Delivery Networks Actually Work

A user in Sydney requests an image hosted on a server in Virginia. Light takes roughly 80ms to make that round trip through fiber, and TCP plus TLS needs several round trips before a single byte of the image arrives — call it 300ms of pure waiting before the download even starts. Now multiply that by the dozens of assets on a page and the millions of users hitting it. That latency, and the bandwidth bill behind it, is the problem a content delivery network exists to erase.

A CDN is a fleet of cache servers spread across hundreds of locations worldwide, sitting between your users and your origin server. It answers most requests from a machine a few milliseconds away instead of forcing every byte to travel back to one datacenter. Get the design right and your origin sees a fraction of the traffic while users get responses that feel instant.

What a CDN actually solves#

"Make it fast" undersells it. A CDN attacks three distinct costs at once.

Latency. By terminating TCP and TLS at an edge node typically under 20ms away (versus 100–300ms transcontinental), it kills the multi-round-trip handshake penalty. It also keeps TCP slow-start effective — a short round-trip time means the congestion window ramps up quickly, so the full response streams at line rate instead of crawling.

Bandwidth and origin load. Edge caches absorb 80–99% of the bytes your origin would otherwise serve. A 10 Gbps origin backed by a CDN behaves like it has effectively unlimited throughput, because the CDN's collective capacity — not your one pipe — faces the crowd.

Resilience. Because the same anycast IP is advertised from 300+ points of presence (PoPs), a volumetric DDoS attack gets spread across the whole fleet instead of drowning one address. The CDN doubles as a TLS offload tier with optimized ciphers, OCSP stapling, and 0-RTT session resumption.

State these as requirements before drawing boxes. A functional requirement says what a caller asks for and gets back; a non-functional one names a bound — latency, freshness, availability, cost. Naming the bounds first stops you from reaching for a familiar tool before you know what has to be true.

Pull vs push: how content gets to the edge#

There are two ways to populate an edge cache, and the choice shapes everything downstream.

Modern CDNs are overwhelmingly pull-based. Object counts have exploded — millions of resized image variants, signed URLs, per-user query strings — which makes pushing every possible object impractical. The cold-miss penalty of pull is real, but tiered caching solves it: misses funnel through a regional parent (an "origin shield") so the origin sees one fetch per object, not one fetch per edge. If you have 200 edges and 1 object, that's the difference between 200 origin hits and 1.

Walkthrough: one request through the edge#

Trace a single request for /static/hero.jpg from a user in Tokyo, with a Cloudflare-style pull CDN in front of an origin in Virginia.

Step 4 is the whole point: exactly one origin fetch served the first request, and every subsequent user in that region pays only the 8ms edge round trip. The immutable directive tells the browser it never needs to revalidate, and the content-hashed filename means a new deploy uses a new URL — so invalidation is a non-event.

Cache headers: telling the edge what to do#

The CDN's behavior is driven almost entirely by response headers. Designing them for a news site with three kinds of content shows the reasoning:

Static assets live for a year because the filename (app.a1b2c3.js) changes when the content does. Article HTML stays cacheable for 60 seconds, then stale-while-revalidate=300 lets the edge serve the stale copy for five more minutes while it refetches asynchronously — the user never waits on the origin — and stale-if-error=86400 keeps the site up for a day even if the origin dies. Comments are private because they personalize per user.

One trap dominates cache-key hygiene: the Vary header. Each value you vary on multiplies the number of cached copies. Vary: Accept-Encoding is fine (a handful of values). Vary: Cookie is a disaster — every distinct cookie becomes its own cache entry, and your hit rate collapses toward zero.

Why "purge by URL" isn't enough#

Cache invalidation at CDN scale is genuinely hard because a single logical object maps to many cached URLs — query-string variants, Accept-Encoding values, device classes. Purging by URL means enumerating all of them, which is intractable for parameterized or personalized content.

The production answer is surrogate keys (also called cache tags). The origin stamps responses with Surrogate-Key: product-123 user-456, and a single PURGE tag=product-123 invalidates every related URL globally — Fastly does this in under 150ms. Two other techniques reduce how often you purge at all: versioned URLs (a new deploy path is born fresh, so nothing needs purging) and stale-while-revalidate (trade a bounded staleness window for zero user-visible invalidation latency).

Capacity: what a Superbowl-scale spike demands#

Numbers expose whether a design is real. Take a live stream: 50 million concurrent viewers at 6 Mbps each.

No single origin serves 300 Tbps, so the architecture is a tree. The origin encodes each segment once and pushes to roughly 10 regional mid-tier caches; each mid-tier feeds around 100 edge PoPs; each PoP runs 20–50 cache nodes facing clients. Live HLS/DASH uses 2–6 second segments with max-age matching the segment duration, so each segment is fetched exactly once per cache node in the tree. Request coalescing is what makes this hold — Varnish's hit-for-pass or Nginx's proxy_cache_lock collapse thousands of simultaneous misses for the same segment into one upstream fetch.

Sizing the edge tier:

The failure mode to engineer against is the thundering herd at each segment boundary, when every player's clock ticks over simultaneously and demands the next segment at once. You smear that spike by staggering client clocks or using chunked CMAF so requests arrive as a stream rather than a synchronized pulse.

Anycast vs DNS routing#

How does a user actually get pointed at the right PoP? Two schools, with different failure modes.

In practice the two converge: Akamai now runs a hybrid — anycast for the fast layer, DNS for coarse region selection, and 301 redirects at the app layer for fine-grained control.

How this shows up in interviews#

CDN questions probe the boundary between a plausible diagram and an operable system. Expect the interviewer to push on cache hit rate first ("your edge is at 40% miss — diagnose it"), which is your cue to talk about Vary misconfiguration, unnormalized tracking params shattering the key space, working sets that exceed edge SSD, and tiered caching to rescue it. From there they escalate: dynamic and personalized content (decompose into public shells plus private fragments assembled with edge compute), invalidation strategy (surrogate keys over URL purges), and capacity under a spike.

The move that reads as senior: pick one request, trace it end-to-end, name the source of truth, then state the one trade-off you'd reverse if the workload changed — for example, dropping stale-while-revalidate if the product cannot tolerate any staleness, and paying the origin-load cost that decision implies.

Further learning#

• What Is A CDN? How Does It Work? — ByteByteGo's whiteboard walkthrough of the edge-caching model.
• What is a CDN? How Do CDNs Work? — Cloudflare's reference explainer on PoPs, caching, and routing.
• Related roadmap topics: Message Queues, Consistent Hashing, API Gateway, and Database Indexing.
• Step through the model interactively on the CDN roadmap topic.

FAQ#

Is a CDN only useful for static files like images and CSS?#

No. Static assets are the easiest win, but modern CDNs cache and accelerate dynamic content too. You decompose a page into a public shell (cached with a long TTL) and private fragments (fetched per request), then assemble them at the edge with edge-side includes or edge compute like Cloudflare Workers. API responses often tolerate short TTLs of 5–30 seconds with stale-while-revalidate, which is fine for feeds and recommendations where bounded staleness is acceptable. For authenticated content, you cache based on signed-URL or JWT claims rather than the full token so cache keys stay bounded.

How do I fix a high cache miss rate at the edge?#

Start with the cache key. A stray Vary: User-Agent or failure to normalize tracking parameters like utm_* can explode one object into thousands of distinct entries, so the same content never gets reused. Next, compare your working-set size to edge storage — if hot objects exceed the node's SSD (roughly 2TB), LRU eviction throws them out prematurely. The structural fixes are tiered caching with a regional shield (funneling misses through one parent cuts origin fetches by around 90%) and consistent-hashing each object to a specific edge node so it lives on one machine instead of being duplicated across dozens.

What is the difference between a pull CDN and a push CDN?#

A pull CDN fetches content from your origin lazily — the first user to request an object at a given edge triggers an origin fetch, and the edge caches the result according to its TTL and headers. A push CDN requires you to upload assets to the CDN ahead of time, giving predictable latency for every user but wasting storage on content nobody requests and complicating every deploy. Nearly all modern CDNs are pull-based because the number of possible objects (image variants, signed URLs, parameterized paths) is far too large to pre-push, and tiered caching already tames the cold-miss cost.

Why can't I just purge a URL to invalidate cached content?#

Because one logical object usually maps to many cached URLs — different query strings, compression encodings, and device variants — so purging by exact URL means knowing and enumerating every one of them, which is impractical for personalized or parameterized content. CDNs solve this with surrogate keys: the origin tags responses with a label like product-123, and purging that single tag invalidates every associated URL globally in well under a second. Versioned, content-hashed filenames sidestep the problem entirely, since a new deploy simply uses new paths that were never cached.

How does a CDN protect my origin during a traffic spike?#

Two mechanisms do the heavy lifting. Edge caches serve the vast majority of requests without ever touching the origin, so a spike in users doesn't translate into a proportional spike in origin traffic. For the requests that do miss, request coalescing (Varnish's hit-for-pass, Nginx's proxy_cache_lock) collapses many simultaneous misses for the same object into a single upstream fetch, so 10,000 concurrent misses become one origin request. Layer a regional shield on top and the origin sees at most one fetch per object across the whole fleet, no matter how large the crowd.