Food Delivery: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Restaurant discovery — browse (~70% of traffic)
User opens app. Location → nearby restaurants + recommendations. Cached ML recommendations, Redis for hot content.
Mobile RTT + auth. Cellular network dominates.
Personalized recs cached. Hit rate 60%+.
Redis geospatial index. Returns 50-100 nearby restaurants sorted by distance.
Optimize: Cache 'restaurants in neighborhood' with 5-10min TTL. Individual restaurant details cached longer.
Personalization + availability + delivery-time-aware ranking.
Optimize: Async rerank: return unranked results fast, replace with ranked when ready. Amortize.
Response with 20-30 restaurants + estimated delivery time each.
Browse is **~100-500ms p99** — ML ranker dominant. **The ETA prediction is critical UX** — users choose based on 'delivery in 25 min' vs '45 min'. ML dispatch must predict this accurately AND consistently or users lose trust.
Bottleneck summary
Food delivery latency is **COMPOUND**: browse (100ms) + order (500ms) + restaurant accept (5s-2min) + driver dispatch (5-30s) + prep time (10-30min) + delivery time (10-30min) = **30-60min end-to-end**. **The user-facing latency budget is dominated by NON-COMPUTATIONAL delays**: restaurant staff, cooking, driving physics. Our infrastructure is <1s combined. **The critical UX pattern**: hide the compound tail via progress UI + WebSocket updates + optimistic states. Never spinner-forever.
Optimization tips (this architecture)
- **Cache ETAs per neighborhood**: 30s-5min TTL. Users see quick 'delivery in 25 min' during browse.
- **ML dispatch with prep-time awareness**: Predict when food is ready + when driver arrives. Match precisely.
- **Parallel dispatch to top-2 candidates**: 60% faster acceptance vs sequential. Race-based dispatch.
- **WebSocket first for driver push**: <100ms delivery. APNS/FCM fallback for backgrounded apps.
- **Idempotency for order commit**: Prevents duplicate charges from double-click.
- **Circuit breaker on Stripe**: Fallback processor when Stripe 5xx > 5%.
- **Restaurant auto-timeout**: Reroute after 60s of no response. Prevents dead orders.
- **Progressive UI**: 'Sending order' → 'Restaurant accepted' → 'Preparing' → 'Driver assigned' → 'Delivering'. Show progress, hide tail.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.