Real-Time Delivery | System Design Interview Handbook

🪜

The Options Ladder

four ways to move a server-side event to a client

The web is pull-based: clients ask, servers answer. Real-time delivery is the set of tricks for reversing that, so the server can push the moment something changes. There are four rungs, each one closer to a true push channel and each one more stateful than the last.

  SHORT POLLING          client asks every N seconds, mostly gets "nothing yet"
    C ─req─▶ S  (empty)   ┐  simple, works everywhere
    C ─req─▶ S  (empty)   ┼  wasteful; latency = poll interval
    C ─req─▶ S  (data!)   ┘

  LONG POLLING           client asks, server HOLDS the request until data exists
    C ─req─────────▶ S    ┐  near-real-time on plain HTTP
              (waits)     ┼  one connection per message; reconnect churn
    C ◀──data── S  reopen ┘

  SERVER-SENT EVENTS     one long-lived HTTP stream, server pushes many events
    C ─req─▶ S            ┐  server → client only, auto-reconnect built in
    C ◀═event═ S          ┼  text only, rides plain HTTP
    C ◀═event═ S          ┘

  WEBSOCKETS             upgrade HTTP once, then a full-duplex TCP channel
    C ⇄ handshake ⇄ S     ┐  bidirectional, low overhead per message
    C ⇄══════════⇄ S      ┼  stateful, needs its own scaling story
    (both push anytime)   ┘

ℹ️

The jump that matters is from polling to persistent connections. Polling reuses the stateless request-response model, so it scales like any HTTP endpoint. SSE and WebSockets hold a connection open per client, which is where all the hard problems below start.

🎯

Choosing Between Them

direction, frequency, and how much complexity you can afford

Reach for the simplest rung that meets the requirement. Three questions settle it: which direction does data flow, how often does it update, and how much connection machinery are you willing to run.

Technique	Direction	Latency	Cost / complexity	Best fit
Short polling	Client pull	= poll interval	Trivial, but wasteful	Rare updates, small scale, quick hacks
Long polling	Server → client	Near real-time	Reconnect churn	Notifications where WebSockets are overkill
SSE	Server → client	Real-time	Low, auto-reconnect	Feeds, live scores, dashboards, notifications
WebSockets	Bidirectional	Real-time	High, stateful	Chat, games, collaborative editing, trading

💡

SSE is the underused answer. If the client only needs to receive — a live feed, price ticks, a progress bar — SSE gives you real-time push over plain HTTP with reconnection handled for you, and none of the WebSocket scaling burden. Don't default to WebSockets just because it's the one everyone names; reach for it when you genuinely need the client to push too (Chat System).

📌

Connection State Is the Hard Part

persistent connections make servers stateful

A stateless HTTP server is easy: any box handles any request. A persistent connection breaks that. The client is now bound to one specific server for the life of the connection, and the system has to answer questions it never had to before.

Which server holds this user? routing to a connection

To push to Bob, you must find the exact server holding Bob's socket. Two options: sticky routing (a load balancer pins Bob to one node) or a connection registry (a shared store mapping user → server, so any node can look up where to send).

Is the connection still alive? dead-connection detection

TCP can stay "open" long after the client vanished (laptop closed, tunnel dropped). Heartbeats — periodic pings — detect the dead ones so you free memory and stop pushing into the void.

Reconnect and resume picking up where it dropped

Connections drop constantly — phones change networks, servers deploy. The client reconnects with exponential backoff and jitter so a mass disconnect doesn't become a synchronized reconnect stampede (Resilience Patterns). On reconnect it sends a last-event-id so the server can replay whatever was missed during the gap, rather than losing it.

⚠️

Sticky routing is the tempting shortcut and a trap at scale: it undermines even load balancing and turns any deploy into a mass reconnect, because every connection on a restarting node drops at once. A connection registry plus a fan-out backbone scales better, at the cost of a lookup on every push.

📈

Scaling to a Million Connections

a dedicated tier and a pub/sub backbone

Each open connection costs memory (buffers, TLS state — order of tens of KB), so a million connections is real RAM and file-descriptor pressure. The standard shape separates the connection-holding tier from everything else, and uses a pub/sub backbone to fan messages out to whichever gateway holds the target.

   millions of clients
        │  │  │  (WebSocket / SSE)
        ▼  ▼  ▼
  ┌───────────────────────────────┐   Connection gateways: a dedicated tier
  │  GW-1     GW-2     GW-3   ...  │   whose only job is holding sockets.
  └───┬─────────┬─────────┬───────┘   Scale this independently of app logic.
      │         │         │
      └─────────┼─────────┘
                ▼
        ┌───────────────┐             Pub/sub backbone (Redis / Kafka):
        │   pub / sub    │            a message for user X is published;
        │   backbone     │            the gateway holding X is subscribed
        └───────┬───────┘             and delivers it. No gateway-to-gateway
                ▼                      mesh needed.
        ┌───────────────┐
        │  app services  │            Business logic stays stateless behind
        └───────────────┘             the gateway tier.

The gateways become stateless in the way that counts: they hold connections but no business logic, so you can add nodes freely. On deploy, you drain a gateway — stop new connections, let clients reconnect elsewhere with backoff — instead of dropping everything at once (Scaling Patterns).

✅

Delivery Over Flaky Links

acks, buffers, and per-channel order

A pushed message can vanish: the connection dropped between send and receive, and the server never knew. "I sent it" is not "they got it." Guaranteeing delivery over an unreliable link needs three pieces.

Piece	What it does
Client acks	The client confirms receipt; unacked messages are considered undelivered and eligible for resend.
Server-side buffer	Hold recent messages per client until acked, so a reconnecting client can be replayed from where it left off.
Per-channel ordering	Sequence numbers within a conversation or topic, so the client can order and dedup even if delivery is out of order.

ℹ️

Ack-plus-resend means a message can arrive twice — the ack got lost, so the server resends something the client already had. That's at-least-once delivery, and the fix is the same everywhere: the client dedups on message ID, making reprocessing harmless (Messaging, Concurrency Control).

📋

Quick Reference

the four techniques on one card

Technique	Latency	Direction	Cost	Best fit
Short polling	= interval	Client pull	Trivial, wasteful	Rare updates, small scale
Long polling	Near real-time	Server → client	Reconnect churn	Simple notifications
SSE	Real-time	Server → client	Low, auto-reconnect	Feeds, dashboards, live scores
WebSockets	Real-time	Bidirectional	High, stateful	Chat, games, collaboration

ℹ️

The one-liners to keep ready. Pick the lowest rung that fits: SSE for server-push-only, WebSockets only when the client must push too. Persistent connections make servers stateful — solve it with a connection registry and a pub/sub backbone, not sticky routing. Scale connections in a dedicated gateway tier you can drain on deploy. Guarantee delivery with client acks, a server buffer, and per-channel sequence numbers, and dedup on the client because delivery is at-least-once.