Instagram: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Feed read — home timeline (~90% of traffic)
User opens app, requests home feed. Cassandra fetches pre-computed timeline (push fanout), ranker scores + orders, CDN serves media URLs.
Mobile RTT + TLS handshake. Cellular network dominates. WiFi faster.
Decode JWT, validate user session, extract feed params.
Read user's timeline (rows pre-populated by fanout writes when followees posted). Timeline holds ~500 recent posts.
Optimize: Compact timeline table with TTL. Old posts age out. Keep hot timeline in memory via row cache.
Hybrid feed: push for normal followees, pull for celebrities (avoid celebrity fanout explosion). Query recent celebrity posts.
Optimize: Batch celebrity queries with MGET. Cache celebrity recent-posts list with 60s TTL.
gRPC to ranker service. Ranker features: engagement history, recency, ML embeddings. GPU inference.
Optimize: Batch inference: send 500 candidates in one call. GPU amortizes cost across batch. Individual scoring is 10-100x slower.
Assemble response JSON with 20-30 posts + signed CDN URLs for media.
Response delivery. Media URLs — client fetches media separately.
CDN serves 480px thumbnails first (progressive), then full-size. Client prioritizes visible viewport.
Optimize: CDN cache hit rate 90%+ for popular content. Serves from edge PoP <100ms globally.
Feed read total is **~200-400ms p99** — ML ranking + Cassandra query dominate. Media loads from CDN in parallel (client-side), so UX perceives sub-500ms full feed. **The critical UX pattern**: ship response before ALL images loaded, let client progressively render. Users tolerate 200ms for feed + 500ms for images better than 1000ms for both.
Bottleneck summary
Instagram latency is **STRONGLY BIMODAL BY WORKLOAD**: feed reads are ML-ranker-dominated (~300ms p99), photo uploads are BANDWIDTH-DOMINATED (2-30s for large photos), story views are CDN-first-frame (sub-500ms). **The universal pattern**: decouple user-perceived latency from actual work. Photo upload returns 'processing' fast → transcoding happens async. Story starts playing with first 2s chunk → rest streams in background. This is how you build 'feels instant' UX on top of slow underlying operations.
Optimization tips (this architecture)
- **Decouple user response from work**: Fast ack + async processing. Never make user wait for transcoding.
- **Client-side progressive rendering**: Ship response before all images. Client fetches in parallel, shows placeholders.
- **Batch ML inference**: 500 candidates in one call. GPU amortizes cost 10-100x over per-candidate.
- **Hybrid push-pull feed**: Push fanout for normal followees, pull for celebrities. Prevents celebrity fanout explosion.
- **Chunked video (2s segments)**: Play first chunk immediately, pre-fetch continuously. Foundation of ABR streaming.
- **CDN edge for media**: 90%+ hit rate. Popular content served from PoP <100ms globally.
- **Resumable uploads (tus)**: Large files can survive network drops. Critical for mobile UX.
- **Placeholder + async publish**: User sees 'processing' immediately. Real media appears in feed 2-30s later without blocking user.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.