Twitter/X timeline: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Timeline read — normal followees only (~80% of traffic)
User follows only normal accounts. Timeline pre-computed via fanout-on-write. Sub-100ms read from Redis.
Mobile/desktop RTT.
JWT validation, feed params extraction.
Fetch pre-computed timeline (sorted list of tweet_ids by time). Redis sorted set.
Optimize: Cap timeline at 800 tweets. Users don't scroll past 100 typically. Trim on write.
Multi-get from Cassandra for 100 tweet contents. Parallel across shards.
Optimize: Cache hot tweets in Redis with 60s TTL. Popular tweets get 10x traffic — worth caching.
Send 100 candidates to ranker. Ranker uses user engagement history + tweet features + freshness.
Optimize: Cache ranked timelines with 30s TTL. Chronological view bypasses ranker (some users prefer this).
Response with 20-30 tweets + media URLs.
Normal timeline read is **~100-300ms p99** — ML ranker + Cassandra multi-get dominate. Redis timeline lookup is ~10ms. **The architectural win**: push-based fanout means the READ is a simple LRANGE. The COST was already paid on write. Same-shape as Instagram feed.
Bottleneck summary
Twitter timeline latency has THREE profiles: **normal timeline (~200ms p99, ranker-dominated)**, **celebrity timeline (~350ms p99, +100ms for pull), **tweet post (~300ms ack + 5-30s async fanout)**. **The critical architectural decision**: hybrid push-pull. Push for normal followees (write cost = O(followers)), pull for celebrities (avoid fanout explosion). Since reads >> writes, this saves massive write-side work at the cost of small read-side latency.
Optimization tips (this architecture)
- **Hybrid push-pull with celebrity carve-out**: Push for followers < 1M, pull for celebrities. The single most important architectural pattern.
- **Batch ML ranker**: Send 150 candidates in one call. GPU amortizes over batch.
- **Cache celebrity recent-tweets**: 30s TTL, 90%+ hit rate. Beyoncé's recent tweets are the most-cached content on Twitter.
- **Ranked vs chronological**: Let user choose. Chronological is cheaper (no ranker). Ranked has higher engagement.
- **Timeline trim**: Cap at 800 tweets per user. Users don't scroll further. Trim on write.
- **Async fanout via Kafka**: Never block user response on fanout. Users tolerate 5-30s tweet-in-feed delay for followers.
- **Spam classifier on GPU**: Sub-100ms budget for post. CPU-based classifiers can't hit this.
- **Cassandra LOCAL_QUORUM**: For tweets. Fanout uses LOCAL_ONE (best effort — occasional missed fanout is acceptable).
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.