Design a Chat System: Real-Time Messaging at WhatsApp Scale

A chat system looks trivial until you count the connections. WhatsApp routinely holds hundreds of millions of live TCP sockets open at once, delivers a message in under a second, and never loses one even when a phone drops off the network mid-send. None of that comes from a clever algorithm. It comes from a handful of deliberate decisions — how a message travels from one open socket to another, how you store two billion messages an hour so "load the last 30" is a single disk seek, and how you keep a monotonic order when the sender retries three times over flaky LTE.

The interview version of this problem rewards the same discipline as the production version: pin down what the system must promise, put real numbers on the load, then pick the smallest architecture that meets both. Let's build it in that order.

What does a chat system actually have to do?#

Start by separating the two kinds of requirements, because they drive different parts of the design.

Functional — send a 1:1 message; send to a group; fetch the last N messages in a conversation; scroll backwards through history; see online/offline presence and typing indicators; get read receipts; receive a push notification when the app is closed.

Non-functional — these are the ones that decide the architecture:

That last row is the honest one. True exactly-once delivery across an unreliable network cannot be guaranteed, so the design aims for at-least-once plus idempotent writes and lets the sequence number clean up duplicates. More on that below.

Capacity estimation: how big is the pipe?#

Numbers turn hand-waving into decisions. Assume 500M daily active users, each sending 40 messages/day, with a payload averaging 100 bytes plus metadata.

Now storage. Each message on disk is closer to 200 bytes once you add channel id, sender id, timestamp, sequence number, and index overhead:

Bandwidth is roughly 2× the write volume, because every message fans out to at least one recipient — so budget 8–10 TB/day of egress, before presence, typing, and receipt traffic on top. Those figures immediately rule out a single relational primary and point at a horizontally partitioned, replicated store with a fleet of stateful connection servers in front. Everything that follows is a consequence of these numbers.

High-level architecture#

The core insight is that a chat server is stateful — it owns the live WebSocket for every user connected to it — but any two users you want to connect are almost never on the same server. So the design splits into three planes:

• Connection layer — a fleet of WebSocket servers, each holding a slice of the live sockets, sitting behind a connection-aware load balancer.
• Routing layer — a session directory plus pub/sub, so a message that lands on server S1 can find its way to server S2.
• Storage layer — a durable message store, a membership store, and per-user read cursors.

Here is the path a single message takes.

If B is offline at step 3, S1 skips the push and instead marks the message pending and enqueues a mobile push notification. B picks it up on reconnect.

Why WebSockets and not HTTP long-polling?#

A chat connection is bidirectional and long-lived, which is exactly what WebSockets are for: one TCP connection, full-duplex, no per-message reconnect overhead. HTTP long-polling forces the client to re-establish a request after every server push, wasting round trips and headers. Server-Sent Events are one-directional (server → client only), fine for a read-only presence feed but useless for sending.

The pragmatic answer in an interview: negotiate WebSockets first, fall back to long-polling for the <1% of networks — restrictive corporate proxies, ancient clients — where the Upgrade handshake fails. You don't design for the fallback; you tolerate it.

How do you route a message across a fleet of stateful servers?#

This is the heart of the design and the part interviewers push on. User A is pinned to server S1, user B to server S2, and neither server knows about the other's sockets. Two components bridge them:

1. Session directory — a Redis-backed map of user_id → {server_id, connection_id}, written on connect and deleted on disconnect. This is service discovery for humans.
2. Per-server pub/sub channel — every chat server subscribes to its own channel. To reach B, you publish to S2's channel and S2 pushes to B's socket.

So the routing at step 3–4 above is: S1 reads the directory (B → S2), publishes the already-persisted message to S2's channel, and S2 delivers. If the directory lookup shows no active session, B is offline — the message stays in the durable store and a push notification goes out. When B reconnects, a sync-from-cursor handshake streams everything after B's last_seen_message_id, so nothing is missed and a second device gets the same backlog without double-firing push pops.

Data model: storing two billion messages an hour#

The access pattern is narrow and predictable — "give me the last N messages in this channel, then let me scroll backwards" — so the store is optimized for exactly that.

Use a wide-column store (Cassandra or ScyllaDB) with:

• Partition key = channel_id — all of a conversation's messages live together.
• Clustering key = message_id, a time-ordered ULID or Snowflake id, sorted descending — so "latest N" is a single seek from the top of the partition.

Two supporting tables keep the hot path clean:

• Membership — keyed by channel_id, plus a reverse index user_id → channel_ids to build a user's sidebar of conversations.
• Read cursors — last_read_message_id per user per channel, stored separately from the message row. Never put the read cursor on the message itself: every participant reading a popular channel would hammer the same row, creating a hot write.

Group chat is the same shape as 1:1 — a group is just a channel with more members. The message store doesn't change; only the fanout at delivery time does.

Ordering and idempotency when the sender retries#

A flaky network means the client will resend a message it isn't sure landed. Handle it with two mechanisms working together.

Idempotency — the client attaches a self-generated client_uuid to every message. The server upserts on (channel_id, client_uuid), so three retries of the same message collapse into one stored row. The client generates the id, so it survives across reconnects.

Ordering — route all writes for a given channel to a single partition leader (consistent hash on channel_id), and have that leader stamp a monotonically increasing sequence number before fanout. Receivers render strictly by sequence. If a client notices a gap in the sequence, it triggers a catch-up sync from its cursor instead of guessing an order. One writer per channel is the price of clean ordering; since traffic is sharded across millions of channels, no single leader is a bottleneck.

Presence and typing without melting the system#

Presence and typing generate far more events than actual messages, so they get their own cheaper treatment.

Presence lives in Redis as ephemeral state with a TTL refreshed by a client heartbeat every ~30s. If the client disconnects, the key simply expires — no explicit "user went offline" write is needed. Critically, you don't broadcast every status change to every contact. You push presence deltas only to users whose UI is actually looking at that contact — the open conversation or the visible sidebar. A user with 2,000 contacts doesn't generate 2,000 fanouts when they come online.

Typing indicators are fire-and-forget over pub/sub with no persistence and a short client-side debounce, so a burst of keystrokes emits one event every few seconds rather than one per character. If a typing event is lost, nothing breaks — the indicator just clears on its own.

The load balancer is not the usual one#

Balancing WebSockets is different from balancing stateless HTTP. Connections are long-lived and sticky — hours, not milliseconds — so the load balancer must support the HTTP Upgrade handshake and pin each TCP connection to one backend for its entire life. You cannot round-robin individual frames. Use an L4 or WebSocket-aware L7 balancer (NLB, Envoy, HAProxy), and watch for uneven load: a few backends can accumulate a tail of long-lived connections while others sit idle. Deploys need graceful drain — tell a backend to stop accepting new sockets while existing ones finish or migrate via a reconnect hint — or every deploy triggers a reconnect storm.

How this shows up in interviews#

Interviewers use "design a chat system" to see whether you reason from requirements or from buzzwords. The strong signals:

• You reach for WebSockets and can justify the fallback, instead of reciting "use Kafka."
• You identify the stateful-server routing problem (A on S1, B on S2) unprompted and solve it with a session directory plus pub/sub.
• You put real numbers on the load — 20B messages/day, ~1M/sec peak, 4–5 PB/year — and let them drive the datastore choice.
• You say exactly-once is impossible and pivot to at-least-once + idempotency keys + per-channel sequencing.
• You treat presence and typing as cheaper, ephemeral traffic rather than durable messages.

The follow-ups usually drill into one deep dive: message ordering, offline delivery, or the storage schema. Pick the partition key deliberately and defend the hot-partition and hot-row traps, and you're demonstrating exactly the judgment the question is testing.

Further learning#

• WHATSAPP System Design: Chat Messaging Systems — Gaurav Sen walks through the connection layer and message routing on video.
• Designing WhatsApp — High Scalability's deep dive into how WhatsApp holds millions of connections per server.
• Practice this topic on YeetCode — step through the chat system design interactively.

Related designs that reuse these patterns:

• Design a News Feed — fanout-on-write vs fanout-on-read, the same delivery trade-off in a different shape.
• Design a Web Crawler — queue-driven fanout and dedup at scale.
• Design an Autocomplete System — low-latency reads with heavy precomputation.
• Design a Unique ID Generator — where the time-ordered Snowflake message ids come from.

FAQ#

Why can't a chat system guarantee exactly-once delivery?#

Because the network can fail at any point, including after the server has written the message but before the sender receives the acknowledgment. The sender can't tell "lost on the way there" from "lost on the way back," so it must retry, which risks a duplicate. The practical answer is at-least-once delivery combined with idempotent writes: the client tags each message with a UUID, the server upserts on (channel_id, client_uuid), and retries collapse into a single stored row. A per-channel sequence number then gives receivers a clean order regardless of how many retries happened.

Why store messages in Cassandra instead of a relational database?#

The workload is write-heavy — around 230,000 messages per second on average, peaking near a million — and the read pattern is narrow: "fetch the last N messages in one conversation, then scroll back." A wide-column store partitioned by channel_id with a time-ordered clustering key puts an entire conversation in one partition, sorted so the latest messages are a single seek. It also scales horizontally across the 4–5 PB/year of storage with replication factor 3 built in. A single relational primary would fall over on both the write rate and the total volume.

How does a message get from one user's server to another's?#

Each user's live WebSocket is pinned to one stateful chat server, and the two people chatting are almost always on different servers. A session directory in Redis maps user_id → {server_id, connection_id}, updated on every connect and disconnect. When server S1 receives a message for a user on S2, it persists the message, looks up the recipient's server in the directory, and publishes to S2's pub/sub channel; S2 pushes it down the recipient's socket. If the directory shows no active session, the recipient is offline, so the message stays durable and a mobile push notification is queued instead.

How do presence and typing indicators avoid overwhelming the system?#

They are treated as cheap, ephemeral events rather than durable messages. Presence lives in Redis with a TTL that a client heartbeat refreshes every ~30 seconds; a disconnect just lets the key expire, so there's no explicit offline write. Updates are pushed only to users whose UI is actively viewing that contact, not broadcast to every contact. Typing indicators are fire-and-forget over pub/sub with no persistence and a client-side debounce, so a burst of keystrokes emits one event every few seconds and a lost event simply clears itself.

What makes load balancing WebSockets different from HTTP?#

WebSocket connections are long-lived and sticky — they can stay open for hours — so the load balancer must complete the HTTP Upgrade handshake and keep each TCP connection pinned to one backend for its entire life. You can't round-robin individual frames the way you would stateless HTTP requests. This calls for an L4 or WebSocket-aware L7 balancer and careful handling of uneven load, since a few backends can accumulate a tail of long-lived connections. Deploys also need graceful drain so existing connections finish or migrate instead of all reconnecting at once.