E-commerce: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Product browse — cached (~80% of traffic)
User browses product page. Data cached in Redis, CDN edge for images. Sub-200ms.
HTML shell + basic product data cached at edge.
Async fetch for stock status + user-specific pricing.
Hot cache — sub-ms Redis GET.
Stock + pricing data delivered.
Product browse is **~100-400ms p99** — CDN-dominated. Dynamic data via async API keeps page interactive fast. **Layered caching (CDN edge + Redis origin) makes browse feel instant** even at Black Friday scale.
Bottleneck summary
E-commerce latency is **STRONGLY DEPENDENT ON EXTERNAL SERVICES**: Stripe (200-1500ms), Fraud service (50-500ms). Our infrastructure (Redis + Vitess) is <100ms. **The atomicity of Redis SETNX/DECR is what prevents overselling** during flash-sales — it's the single most important pattern in inventory-critical e-commerce. Queue-based flash-sale flows (10M users → 10K/60s release) prevent site collapse.
Optimization tips (this architecture)
- **Redis SETNX/DECR for inventory**: Atomic ops prevent overselling. Sharded by product_id.
- **Idempotency key on checkout**: Client-supplied UUID + Redis SETNX with 24h TTL. Prevents duplicate charges.
- **Circuit breaker per processor**: Stripe → Adyen fallback after 3 failures.
- **Queue-based flash-sale**: Waiting room releases 10K users/60s. Prevents site collapse during Black Friday.
- **Layered caching**: CDN edge (HTML) + Redis (product data) + Vitess (orders). Sub-100ms browse.
- **Fraud score cache**: 5min TTL for repeated users. New users pay full latency.
- **Async fulfillment via Sidekiq**: Email, warehouse notifications, tax calc — never on user-facing path.
- **Vitess sharded by user_id**: Order queries stay single-shard. Cross-shard analytics uses BigQuery.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.