YouTube: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Popular VOD — GGC hit (~90% of streams)
User picks popular video. Metadata from Vitess, video from ISP-local GGC. Sub-1s time-to-first-frame.
Client requests video metadata + player initialization.
Query for title, thumbnail, description, ad slots, monetization info.
Optimize: Cache hot video metadata in Redis with 5min TTL. Trending videos see 100x load.
Google Ads server selects ads via real-time bidding. <50ms budget.
Response delivered.
GGC inside ISP — sub-20ms typical. Similar to Netflix Open Connect.
First segment ~1MB. Bandwidth-limited.
H.264/VP9/AV1 hardware decode. TTFF visible to user.
GGC hit path is **~500-1000ms TTFF**. Ads add ~100ms — necessary for monetization but tail-heavy. **YouTube pre-rolls ads** because ad tail latency (100-300ms) is hidden during video pre-buffer.
Bottleneck summary
YouTube VOD is **GGC-DOMINATED** (like Netflix): ISP-local caches serve 90%+ of traffic sub-1s. Live streams achieve 3-6s latency via LL-HLS. Content ID adds significant upload-to-publication latency (30s-30min) but is legally required. **The critical architectural pattern**: Google Global Cache eliminates cross-region hops for 90%+ of traffic — this is the same insight as Netflix Open Connect.
Optimization tips (this architecture)
- **GGC inside ISP networks**: 15,000+ caches globally, similar to Netflix OCA. Sub-20ms first-byte.
- **LL-HLS CMAF partial segments**: 200ms chunks vs 2s. Start playback 10x faster for live streams.
- **Real-time bidding for ads**: <50ms budget. Batch multiple ad slots in one call.
- **Content ID GPU fingerprinting**: Batch fingerprints, cache reference DB shard-local.
- **Resumable uploads**: tus protocol survives network drops during long uploads.
- **Hardware decode**: VP9/AV1 with hardware acceleration where available. H.264 as fallback.
- **Pre-roll ads during pre-buffer**: Hide ad latency behind video buffer warm-up.
- **WebRTC for sub-1s live**: Use for smaller streams (100K concurrent). Reserve LL-HLS for larger.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.