URL Shortener: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Edge cache hit (~70% of requests)
User's redirect request hits a CloudFront PoP within 50ms of them. Cache is warm. Response comes straight from edge — never touches our origin infrastructure.
DNS resolution + TCP handshake + TLS handshake + HTTP request send. Nearest CloudFront PoP is typically 5-15ms away for urban users, 30-80ms for rural/emerging markets.
Optimize: TLS session resumption (0-RTT via TLS 1.3) cuts second-request latency by 40%. HTTP/2 connection reuse eliminates handshake for subsequent redirects.
CloudFront edge server checks its in-memory KV store for the short_id. Hot keys served from LRU cache; slightly-warm keys from local SSD.
Optimize: Cache TTL 5 minutes with stale-while-revalidate — even during cache misses, we serve stale content while async-refreshing.
Encode 302 redirect response with Location header. Response is tiny (~200 bytes) so serialization is negligible.
Same network path as request, in reverse. TCP ACKs + response delivery. Client browser then initiates request to the long URL (separate transaction from our perspective).
Edge cache hit is dominated by NETWORK LATENCY (round-trip time to nearest PoP). Application logic + cache lookup = <2ms combined. If your users see slow autocomplete, look at DNS + TLS + geography FIRST, application code SECOND.
Bottleneck summary
The dominant latency source depends on your CACHE HIT LOCATION: edge hit (70% of traffic, <15ms), origin Redis hit (25%, <50ms), or MySQL fallback (5%, <100ms). Networks dominate everywhere — application code + cache lookups are <2ms combined. **This is the recurring architectural truth: at scale, latency is 80% network + 15% database + 5% application.** Every 1ms of application optimization is worth 100μs of end-to-end latency. Every 10ms of network reduction (regional origin, edge PoP) is worth 10ms.
Optimization tips (this architecture)
- **PoP density first**: Add edge PoPs near underserved regions before optimizing application code. 1 new PoP in São Paulo can cut p99 for 200M users by 100+ ms.
- **Cache aggressively at multiple tiers**: Edge (5 min TTL) + Redis (5 min TTL) + client-side (1 day for own short URLs). Layered caching amortizes miss cost.
- **Regional origin placement**: 3-region deployment (US-East + EU-West + APAC-Tokyo) turns cross-region misses from 180ms → 30ms for 90% of the world.
- **TLS 1.3 + HTTP/2**: 0-RTT resumption + connection reuse eliminate handshake latency for repeat visitors.
- **MySQL tuning**: innodb_buffer_pool_size at 70-80% of RAM, ProxySQL transaction mode, SSD gp3 minimum. Index EVERYTHING that appears in WHERE clauses.
- **Redis Cluster**: Never a single-node Redis at 1M RPS+. 16 shards × RF=1 for HA. Cluster-aware clients eliminate proxy hops.
- **Measure p99 AND p99.9**: The 0.1% tail is where user churn happens. p99 optimization often IGNORES the tail because averages hide it.
- **Amara Paper**: 100ms of latency = 1% revenue lost. At $500M ARR, 10ms saved = $5M/yr recurring. Latency is a business metric, not a technical metric.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.