WhatsApp: Junior → Architect Evolution
The mandated interactive flow. Step through each rung, ask what breaks FIRST, weigh the options, and defend the chosen architecture. This is how architectural thinking is learned — not by reading a fixed design, but by tracing how it evolves under growth pressure.
Scale Evolution Timeline
Step through 6 architectural rungs, from 10K RPS to 1B RPS. At each rung, ask: what will break FIRST? Why? What options exist? Which one do we pick — and what are we accepting?
This is the reasoning cycle that separates a Junior developer ("here's an architecture") from an Architect ("here's why this architecture, why now, and what breaks next").
10K RPS — XMPP over TCP + Erlang + Mnesia
1. Current architecture
Where we are before growth pressure
1 × Erlang node running FreeBSD + XMPP server (based on ejabberd). Mnesia (Erlang's in-memory distributed DB) holds user state + presence. Messages persist to disk. TCP-based XMPP protocol — persistent socket per user. No E2E encryption (server sees plaintext).
2. Growth trigger
What changed — the traffic/data force
MVP launch. 100K users, mostly on Nokia/BlackBerry mobile devices. Peak 10K messages/sec. Users on unreliable 2G/3G networks — protocol must handle intermittent connectivity.
3. Bottleneck — what breaks FIRST?
The component that saturates as growth arrives
None yet — Erlang + XMPP is genuinely enough
Erlang was built for telecom (Ericsson): millions of lightweight processes, message-passing concurrency, hot-swap deploys. A single Erlang node handles 1M+ concurrent TCP connections at ~2GB RAM. Mnesia + XMPP is a well-worn combination. p99 message delivery: 100-500ms on 2G/3G.
Erlang node RAM 8GB (2M concurrent connections), XMPP CPU 30%, message-delivery p99 300ms (network-dominated), Mnesia disk write rate 2 MB/s
4. Options — what could we do?
Alternatives an architect must consider before picking
- + Erlang's actor model is perfect for messaging (1 process per user)
- + 1 node handles 1M+ concurrent connections
- + Hot-swap deploys = zero downtime updates
- + Team of 5 engineers can run this
- − Single point of failure — plan for FreeBSD kernel panic
- − No E2E encryption (2010 didn't require it)
- − XMPP protocol is chatty — bandwidth expensive on 2G
5. Chosen
The specific decision we're making
Do nothing — Erlang + XMPP is right at 10K RPS
6. Trade-offs
What we're explicitly accepting to move forward
- Accept single node SPOF (rare — Erlang has 9-nines reliability record)
- Accept no E2E encryption (users trust WhatsApp servers with plaintext)
- Accept XMPP protocol overhead on 2G networks
- Accept team knowledge of Erlang (harder to hire than JVM)
7. New architecture
The system after this decision — headroom for the next 5-10x
1 Erlang node + Mnesia + XMPP. 5 engineers maintain. Total cost: $100/mo. WhatsApp's actual 2010 architecture.
8. Next bottleneck — what will break at the NEXT rung?
This is the seed of the next rung
At ~100K RPS with 10M+ users, single Erlang node hits connection ceiling (2-3M concurrent connections). Also E2E encryption becomes mandatory (Snowden 2013 changed the industry). Federated server topology + Signal Protocol — L5 shape.
What comes next in your Junior → Architect journey
You've traced 6 rungs of WhatsApp evolution. Now try the same reasoning cycle on a system you don't know yet — pick from the Systems catalog and answer the same questions: current arch → growth trigger → bottleneck → options → chosen → trade-offs → new arch → next bottleneck. That is architecture.