Design YouTube: Upload Pipeline, CDN, and Video at Planet Scale
How to design YouTube in a system design interview — the transcoding DAG, ABR streaming, multi-tier CDN, storage tiering, view counting, and capacity math.
This article has an interactive companion
Don't just read it — step through it interactively, one state change at a time.
Netflix and YouTube look like the same problem — pixels on a screen, bytes over a wire — but one choice splits them. Netflix ships a catalog it encoded in advance; YouTube accepts 500 hours of raw video every minute from anyone with a phone. That write amplification from user-generated content, not the player, is the heart of the design.
The second shaping fact: watch time is skewed. Roughly 10% of videos drive 90% of views, and the rest sit in a cold long tail — so storing every rendition on fast disk goes bankrupt. The design is two arguments stitched together: how to ingest a firehose of uploads, and how to serve a power-law of reads without keeping the tail hot. This walks the standard interview rubric with numbers that survive a follow-up.
Functional and non-functional requirements
The functional surface is small: a creator uploads a video and it becomes playable in multiple qualities; a viewer streams video that adapts to the network and sees a view count. Comments and monetization are side systems. The non-functional bounds are where the design lives.
| Requirement | Target | Drives |
|---|---|---|
| Playback start | ~2s to first frame | Edge-cached pre-encoded segments |
| Upload durability | Never lose an accepted upload | Resumable chunked upload, replicated blobs |
| Watch availability | Reads must not fail | Multi-tier CDN, stateless serving, bitrate fallback |
| Transcode latency | Watchable within minutes | DAG pipeline, cheap codec first |
| Storage cost | Bounded despite exabyte growth | Hot/cold tiering, erasure coding, JIT re-transcode |
| View count | Approximate live, exact eventually | Streaming approximation + nightly reconciliation |
Consistency is deliberately weak almost everywhere — a minute-stale view count or an hour-stale recommendation row is fine. That relaxation is the budget you spend to keep reads cheap and writes async.
Capacity estimation: the numbers that force the architecture
Do this math out loud — it's the argument for every expensive decision later. Public-scale figures: ~2B monthly users, ~500 hours uploaded per minute, ~1B watch-hours per day. Storage first — convert uploads to hours per day, then multiply by bytes per hour:
500 hr/min × 60 = 30,000 hr/hr = 720,000 hr/day
per source hour: ~1 GB mezzanine + ~3 GB full rendition ladder ≈ 4 GB
720,000 hr × 4 GB = ~2.9 PB/day ≈ ~1 EB/year
with 3× replication: ≈ ~3 EB/year of net-new storageNow peak egress. Little's Law gives average concurrency directly — divide watch-hours accrued per day by the hours in a day:
1B watch-hours/day ÷ 24 h/day ≈ 41.7M concurrent streams (average)
peak ≈ 2× average ≈ ~85M concurrent streams
at ~3 Mbps per stream: 85M × 3 Mbps ≈ 250 Tbps at peakSit with ~250 Tbps — no commercial CDN sells that egress at a survivable price. That is why YouTube runs its own CDN (Google Global Cache) embedded in ISP networks rather than renting from Akamai or Cloudflare — the same logic behind Netflix's OpenConnect. The write math justifies tiering; the read justifies owning the edge.
High-level architecture: two paths that barely touch
Split the system into an upload path and a watch path and the diagram writes itself. The upload path is write-heavy and asynchronous. A resumable chunked upload lands the raw file directly in blob storage (S3/GCS-style); completion drops an event on a durable queue (Kafka/PubSub); a transcoding orchestrator consumes it, produces the rendition ladder, and marks the video ready. The uploader gets "processing" immediately and a webhook when renditions go live — never blocking on encoding.
The watch path is read-heavy and near-stateless. A playback API authenticates, reads metadata, and returns a manifest URL plus a signed segment base URL; from there the client talks only to the CDN and owns quality selection.
Decoupling is the point: a transcoding backlog can't take down playback, and a viral spike can't slow uploads — they share almost no synchronous infrastructure.
The transcoding pipeline: why a DAG, not a monolith
Encoding a raw upload is not one job — it's a fan-out of independent jobs, which is what a DAG orchestrator (Argo/Airflow-style) is for:
| Stage | Work | Runs how |
|---|---|---|
| Probe | ffprobe reads container, resolution, codec, duration | Fast, blocking |
| Split | Chop source into GOP-aligned chunks | Fast |
| Thumbnails + audio | Extract poster frames, separate audio | Parallel |
| Encode ladder | 240p → 4K, each as H.264, VP9, AV1 | Massively parallel |
| Package | Wrap renditions into DASH/HLS segments + manifest | Per rendition |
| Publish | Write locations to metadata, flip status to ready | Final |
The DAG earns its keep four ways: it parallelizes independent resolutions and codecs across a fleet; it retries only the failed shard, so a dead 4K AV1 chunk doesn't force redoing 240p H.264; it caches the mezzanine so stages don't re-decode the source; and it schedules by cost — cheap H.264 immediately so the video is watchable in minutes, expensive AV1 deferred to spot/GPU capacity. A monolith does none of that: one failure restarts everything and the viewer waits for the slowest codec.
API design
The play-time APIs are intentionally thin so the CDN does the heavy lifting:
POST /upload/init -> { uploadId, chunkUrls[] } # resumable session
PUT /upload/{id}/chunk -> 200 # idempotent chunk PUT
POST /upload/{id}/complete -> { videoId, status: "processing" }
GET /watch/{videoId} -> { manifestUrl, segmentBaseUrl, poster, viewCount }
POST /view -> body: { videoId, sessionToken, positionMs }/upload/init hands back pre-signed chunk URLs so bytes go straight to blob storage. /watch is the only origin call in a viewing session — after it, every segment fetch hits the CDN. /view is fire-and-forget, letting the counter be eventually consistent.
Data model and storage tiering
The metadata is modest and relational-ish: a videos row (id, uploader, title, duration, status, view count) and a renditions table mapping each (videoId, resolution, codec) to a storage location and tier. The heavy asset lives in blob storage — tiering is where the power-law pays off:
| Tier | Backing | Holds | Access |
|---|---|---|---|
| Hot | SSD blob, 3-region replicated | Recent + top-K popular | Low-latency origin fetch |
| Warm | HDD standard storage | Fading mid-tail | Slower, still online |
| Cold | Erasure-coded archival (Glacier/Coldline) | Source + key renditions only | Minutes; may JIT re-transcode |
A background popularity scorer — view velocity with recency decay — moves objects between tiers nightly. For the cold tail, keeping only the source plus a rendition or two and re-transcoding on the rare request beats storing the full ladder forever; the metadata service records which renditions exist where so playback can fall back or trigger just-in-time encoding. Erasure coding in the cold tier, rather than 3× replication, cuts durability overhead from 200% to ~40–50%.
Deep dive: the CDN and the viral cold-start problem
Serving hundreds of Tbps means a multi-tier cache. Edge PoPs hold hot segments under LRU; behind them, regional caches act as origin shields that absorb edge misses so the blob store sees almost nothing. Anycast sends each viewer to the nearest healthy PoP.
The hard case is a cold video that suddenly goes viral — a segment nobody cached, requested by millions at once. Two mechanisms save you. Request coalescing at the shield collapses N concurrent misses for one segment into a single origin fetch, so a thundering herd becomes one read. And consistent hashing with bounded loads spreads a hot video's segments across nodes instead of hammering the one its key maps to. A promotion service pre-warms regional tiers once request rate crosses a threshold; the player rides a lower bitrate until higher renditions warm.
Adaptive bitrate (ABR) makes this work on real networks. The encoder emits the ladder as a manifest of bandwidth, codec, and segment URLs (2–6s each), but the client — Shaka, ExoPlayer, hls.js — owns switching via an algorithm like BOLA, keyed on throughput and buffer level. The server stays dumb and stateless, caching perfectly while the client reacts to a dip without a round trip.
Deep dive: counting views without lying or being fooled
A view counter has three conflicting demands: fast enough to show live, exact enough to trust eventually, hard to game. The answer is a lambda-style split. The client emits an event only after a real watch threshold (~30s or 50%), carrying a short-lived token from playback start so a replay can't inflate it. Events flow into Kafka; a streaming job (Flink) deduplicates by (userId, videoId, session), applies fraud heuristics — IP/device velocity, autoplay chains — and bumps an approximate counter in Redis for display. Overnight, a batch job recomputes the exact, fraud-filtered count from the raw event log (BigQuery) and reconciles it into metadata — live-ish instantly, exact by morning.
Deep dive: what changes for live streaming
VOD assumes the whole file exists before anyone watches. Live inverts that — glass-to-glass latency under ~5s while the video is still being produced. Ingest switches from batch HTTP upload to an RTMP/SRT/WebRTC push into a regional server that transcodes in real time on GPUs. Instead of 6-second segments it emits LL-HLS or LL-DASH with CMAF chunks of ~200ms, and the CDN must support chunked transfer encoding so players get sub-segment data before a full segment is written. The trade-offs are real: shorter chunks mean less ABR headroom and lower cache efficiency. The DVR window is written to blob storage asynchronously, so the stream becomes an ordinary VOD replay once it ends.
How this shows up in interviews
Lead with the capacity math — the exabyte-per-year storage and hundreds-of-Tbps egress justify owning a CDN and tiering storage. Then trace both paths and expect probes on the viral thundering herd, the two-system view counter, and where ABR logic lives.
You can explore this design interactively on YeetCode to reinforce the flow.
Further learning
- Design YouTube: System Design Interview — NeetCode's whiteboard walkthrough of the same problem.
- Design YouTube w/ Ex-Meta Staff Engineer — Hello Interview's deep dive with a staff-level lens.
- Related roadmap designs: Design Instagram, Design Twitter, Design Uber, and Design Google Maps.
FAQ
Why is transcoding modeled as a DAG instead of one big job?
The economics only close when work runs in parallel and failures stay local. A single upload spawns dozens of encodes — every resolution crossed with every codec, plus probing, thumbnails, and audio — with no reason to wait on each other. A dependency graph packs them across a fleet, resumes from one broken shard rather than rewinding the whole encode, and reuses a single decoded intermediate. It also splits time-to-watchable from time-to-optimal: publish a cheap codec the instant it lands, run the pricey high-compression pass in the background.
What storage and egress does a YouTube-scale service require?
On the write side, ~720K source-hours land daily; at ~4 GB per hour across intermediate plus ladder you accrue roughly an exabyte a year before replication, three with it. Reads are scarier: with about a billion watch-hours a day, Little's Law puts average concurrency near 42 million streams and peak near double that. At a few Mbps apiece, ~85 million streams push on the order of 250 Tbps — which forces serving off rented transit and onto caches inside ISP networks.
Client or server — where does bitrate selection actually live?
On the player, by design. The backend only publishes a menu — a manifest enumerating each quality's bitrate and codec alongside short, independently fetchable chunks — and serves whatever file gets asked for. Which quality to pull next belongs to the device, because only the device sees its own throughput and buffer occupancy moment to moment. That keeps every server response a static, cacheable object and lets playback recover from a bandwidth drop on the very next chunk.
Make it stick: run this one yourself
Don't just read it — step through it interactively, one state change at a time.