Design Instagram: Feed Fanout, Stories, and Ranking at Scale
Design Instagram end to end — capacity math for 500M DAU, hybrid feed fanout, Stories with 24h TTL, sharding, and ML ranking within a 200ms budget.
This article has an interactive companion
Don't just read it — step through it interactively, one state change at a time.
Instagram looks simple from the outside: post a photo, scroll a feed, tap through some Stories. The hard part is that the feed a user sees in 200 milliseconds is assembled from posts scattered across hundreds of accounts, half of which they follow, one of which might have 100 million followers. Every design decision in this problem bends around that one asymmetry.
The interview is not asking you to draw Instagram. It's asking you to pick a partition key, decide who pays the cost of a celebrity's post — the writer or the reader — and defend a latency budget with numbers. Get the fanout model right and the rest of the design falls into place.
This walkthrough follows the standard rubric: functional requirements, capacity math, high-level architecture, API and data model, then deep dives on the three parts that actually break at scale — feed generation, Stories, and ranking.
What the system has to do
Start by fencing the scope. A complete Instagram is enormous; an interviewable Instagram is four features done well.
- Post a photo. Upload media, store it durably, generate thumbnails, attach a caption and metadata.
- View a home feed. A ranked, paginated list of recent posts from accounts you follow, plus injected recommendations.
- Follow accounts and view profiles. The social graph, plus a reverse-chronological grid of a user's own posts.
- Stories. Ephemeral posts that expire after 24 hours, shown as a tray of unwatched circles at the top of the app.
Explicitly park the rest: Reels, DMs, comments threading, and ads are separate systems you'd mention and defer. Naming what you're not building is how you show scope control instead of freezing on the size of the problem.
The non-functional promises matter more than the feature list. The home feed must load in under 200ms at p99 or the app feels broken. Posts must be durable — a lost photo is unacceptable. But the feed itself can be eventually consistent: if your friend's new post shows up three seconds late, nobody notices or cares. That single relaxation is what makes asynchronous fanout legal.
Capacity estimation with real numbers
Assume 500M daily active users. Each views ~50 posts/day and uploads ~0.2 photos/day. Those two ratios drive every sizing decision.
Write path. 500M × 0.2 = 100M photos/day. Spread over 86,400 seconds that's ~1,150 writes/sec on average, with a peak of 3,000–5,000 writes/sec during evening spikes. Each photo is stored as one processed original (~500KB) plus roughly three thumbnail sizes (~200KB total), so budget ~700KB per post. That's 100M × 700KB ≈ 70 TB/day, or ~25 PB/year raw — call it 75 PB/year once you apply 3× replication.
Read path. 500M × 50 = 25 billion feed views/day ≈ 290K/sec average, peaking near 600K/sec. Each feed call fans out into a cache read, a metadata read, and a ranking pass, so the feed service itself needs to be sized for 1–2M requests/sec of internal capacity to absorb bursts.
The takeaway that shapes the architecture: this is a system with a 500:1 read-to-write ratio. Reads dominate by three orders of magnitude, which justifies precomputing feeds and spending write-time work to make read-time cheap.
High-level architecture
Keep the synchronous request path short. When a user uploads a photo, the API does the minimum: authenticate, validate, push the media to blob storage, write one metadata row, and return 201 Created. Everything else — thumbnail generation, feed fanout, search indexing, notifications — happens off a durable queue after the response is sent.
Client
│
▼
Load Balancer ──► Stateless API tier
│ (upload → blob store, write metadata row, enqueue)
▼
┌──── Kafka (post events) ────┐
▼ ▼ ▼
Fanout worker Media pipeline Search indexer
│ (thumbnails) (hashtags, geo)
▼
Redis timelines (materialized feeds)The stateless API tier scales horizontally behind the load balancer. Blob storage (S3 or a Haystack-style store for small files) holds the images; a sharded relational store holds metadata; Redis holds the precomputed per-user timelines that make feed reads fast. Kafka is the spine that decouples the fast write from the slow fanout.
API and data model
The public API is small and cursor-based — offset pagination breaks when new posts shift every row.
POST /v1/media → { upload_url } # pre-signed, direct to blob store
POST /v1/posts → { post_id } # caption + media_id
GET /v1/feed?cursor=<c> → { posts[], next } # ranked home feed page
GET /v1/users/{id}/posts → { posts[], next } # profile grid, reverse-chron
POST /v1/follow → { ok }
GET /v1/stories/tray → { authors[], seen[] } # active-story authorsThe metadata schema is deliberately narrow. The core posts table holds post_id, user_id, caption, media_url, created_at, like_count. Shard by user_id hash so every one of a user's posts lands on the same shard — that makes the profile grid a single-shard scan instead of a scatter-gather.
The post_id is a Snowflake-style 64-bit ID: a timestamp prefix, a shard/machine ID, and a per-ms sequence number. Because time is the high bit, IDs sort chronologically, which turns feed pagination into a simple WHERE id < cursor ORDER BY id DESC LIMIT n. Secondary access patterns — hashtag search, location search — do not get secondary indexes on the primary DB; they go to a separate inverted index (Elasticsearch) so the write path stays cheap.
The feed fanout problem
This is the crux of the whole design. When someone posts, how does it reach their followers' feeds? Two pure strategies, each broken at one end.
| Strategy | How it works | Write cost | Read cost | Breaks on |
|---|---|---|---|---|
| Push (fanout-on-write) | Copy the post into every follower's materialized feed at post time | O(followers) per post | O(1) — feed is precomputed | Celebrities: one post = 100M writes |
| Pull (fanout-on-read) | Assemble the feed at read time by querying followed users' recent posts | O(1) per post | O(following) per read | Active users with 1000s of follows; every scroll re-queries |
Neither wins alone. Push makes reads instant but a single Cristiano Ronaldo post would trigger 100M+ timeline writes. Pull makes writes trivial but forces an expensive scatter-gather on every one of those 600K reads/sec.
Instagram uses a hybrid. Ordinary accounts are pushed: when @maya with 800 followers posts, a fanout worker writes her post_id into 800 Redis timelines — cheap, and her followers get O(1) reads. Accounts above a celebrity threshold (say, >1M followers) are not fanned out. Their recent posts are pulled at read time and merged into the requesting user's timeline. So a feed read = "read my materialized Redis timeline (from pushed users) + query the handful of celebrities I follow + merge and rank." This caps fanout amplification and keeps the p99 under 200ms for the 99% of users who follow zero or few celebrities.
Ranking within the latency budget
A chronological feed was Instagram's past; today the feed is ranked by an ML model. The problem: you might have 1,000 candidate posts and a deep neural network that's far too expensive to run on all of them inside 200ms.
The answer is a two-stage funnel. Stage one, candidate generation, cheaply pulls ~500–1,000 posts from sources (following, interests, Explore) in ~30ms. Stage two, the heavy ranker, scores just those candidates with a DNN on a dedicated inference fleet (TF Serving or Triton) using batched gRPC.
The features that feed the model come from an online feature store (Redis/Rockset) answering in p99 ~5ms. User and item embeddings are precomputed offline and cached per session; only fast-changing interaction features are computed online. A representative budget: ~30ms candidate generation, ~50ms feature fetch, ~40ms scoring — leaving comfortable headroom for network and render inside the 200ms envelope.
Stories, seen-state, and view counts
Stories add two twists: everything expires in 24 hours, and a single active user might have 200+ unwatched stories across 300 followed accounts.
Stories live in a KV store (Cassandra) as a per-author list with a 24-hour TTL column — expiry is the storage engine's job, backed by a compaction sweep that purges expired media. A per-user "recent active authors" index tracks which followed accounts currently have a story in window, so building the tray is: fetch active-author IDs, filter by seen-state, rank by recency and affinity. Media sits in blob storage behind signed URLs; the service only returns metadata.
Seen-state is where consistency gets interesting. The viewer's own "have I watched this?" state must be immediately consistent on their device, so it's a per-user Redis roaring bitmap keyed by story_id, written synchronously on view. Global view counts, however, can lag. Each view emits a (viewer, story, timestamp) event to Kafka; a Flink job aggregates counts into a counter store with seconds-level lag. The author's "seen by" list is a bounded list capped at ~50 recent viewers — storing all 10M viewers of a celebrity's story is pointless, so the count stays exact while the list is truncated.
Notifications follow the same asynchronous shape: like/comment/follow events go to Kafka, and a notification service applies coalescing ("5 people liked your post" instead of 5 pushes), per-user rate limiting, and quiet hours before batching to APNs/FCM within each provider's rate caps.
How this shows up in interviews
The moment you say "fanout," the interviewer will push on the celebrity case — have the hybrid threshold ready before they ask. Other reliable follow-ups: what's the partition key and does it create hot shards (a viral post is a hot shard); how do you paginate a feed that's changing under you (cursor on the Snowflake ID); is the feed consistent or eventual (eventual, and say why that's fine); and how the design changes at 10× load (more Redis shards, deeper celebrity list, regional feed caches).
The winning move is to walk one request end to end — @maya posts, the worker fans out to 800 timelines, her follower's next feed read merges pushed posts with pulled celebrity posts and ranks them — then name the one trade-off you'd reverse if the workload shifted. That narrative beats any static box diagram.
Further learning
- Designing INSTAGRAM: System Design of News Feed — Gaurav Sen — the classic walkthrough of feed generation and the push/pull decision.
- Practice the full rubric interactively: Practice this topic on YeetCode.
- Related designs that reuse these patterns: Design Twitter (the closest cousin — same fanout problem), Design Netflix (blob storage and CDN at scale), Design Uber, and Design Google Maps.
FAQ
Does Instagram use push or pull for the feed?
Both — it's a hybrid keyed on follower count. Ordinary accounts are pushed: their post is copied into every follower's materialized Redis timeline at write time, so those followers get O(1) feed reads. Accounts above a celebrity threshold (roughly 1M+ followers) are excluded from fanout because a single post would trigger 100M+ writes; their recent posts are instead pulled at read time and merged into each requesting user's feed. The hybrid caps write amplification while keeping p99 feed latency under 200ms for the vast majority of users.
How much storage does Instagram's photo workload need?
At 500M DAU posting ~0.2 photos/day, that's 100M new photos daily. Budgeting ~700KB per post (a ~500KB processed original plus ~200KB of thumbnails) gives ~70 TB/day of new data, which is about 25 PB/year raw — or roughly 75 PB/year once you apply 3× replication. Blobs go to object storage while only the small metadata rows live in the sharded relational database.
How do you shard the metadata database?
Shard by user_id hash so all of a user's posts colocate on one shard, turning the profile grid into a single-shard scan rather than a scatter-gather. Post IDs are Snowflake-style 64-bit values with the timestamp in the high bits, which makes them globally sortable and lets feed pagination use a simple cursor on the ID. Secondary lookups like hashtags and locations go through a separate inverted index (Elasticsearch) instead of secondary indexes on the primary store, keeping the write path fast.
How does feed ranking stay under the latency budget?
With a two-stage funnel. A lightweight candidate generator pulls 500–1,000 posts from following, interests, and Explore in about 30ms, then a heavy DNN ranker scores only those candidates on a dedicated inference fleet using batched gRPC. Features come from an online feature store answering in ~5ms p99, with user and item embeddings precomputed offline and only interaction features computed live. A typical split is ~30ms candidate generation, ~50ms feature fetch, and ~40ms scoring — well inside a 200ms envelope.
How is "seen by" tracked for Stories at 500M DAU?
The viewer's own seen-state lives in a per-user Redis roaring bitmap keyed by story ID, written synchronously on view so the UI is instantly consistent on that device. Global view counts are eventually consistent: each view emits an event to Kafka, and a Flink job aggregates per-story counts into a counter store with seconds-level lag. The author's visible "seen by" list is bounded to about 50 recent viewers because storing all 10M viewers of a celebrity's story is wasteful — the aggregate count stays exact while the displayed list is truncated.
Make it stick: run this one yourself
Don't just read it — step through it interactively, one state change at a time.