Ride Sharing: Latency Waterfall
Break down end-to-end latency by hop and percentile. Understand where the p99 tail comes from — DNS, network, cache lookup, database query, serialization. Real requests have wildly different paths depending on cache-hit location.
Latency Waterfall
Break down end-to-end latency by hop (network, application, database, cache) and percentile (p50/p90/p99/p99.9). Real requests have wildly different paths depending on cache-hit location — pick a scenario to see the full waterfall.
Amara Google 2009: every 100ms of latency = 1% revenue lost. Understanding WHERE the tail comes from is the difference between random optimization and targeted engineering.
Rider requests ride — full match flow (~10% of traffic)
Rider taps 'Request Ride'. DISCO finds candidate drivers within 3-5 hexes, ranks them by ETA + acceptance rate, sends dispatch to top candidate.
Mobile RTT (LTE/5G) + TLS 1.3 handshake (resumable) + ALB routing to nearest healthy app. Cellular network adds jitter.
Optimize: Persistent WebSocket connections avoid handshake on each request. Regional deployment (US/EU/APAC) cuts RTT for non-US users.
Deserialize JSON, extract pickup lat/lon, compute H3 hex_id at resolution 9 (~150m diameter). Uber H3 library is fast.
Same-region TCP hop from app tier to DISCO tier.
Query current hex + 6 neighboring hexes in parallel. Each shard responds with driver_ids in that hex. Parallel MGET across 2-3 shards typical.
Optimize: Prefer LOCAL_QUORUM if Redis Cluster spans AZs. Batch MGET into cluster-aware pipeline to amortize latency.
For each candidate: predict ETA (Uber DeepETA at L7, simpler haversine at L6), fetch acceptance rate + rating from Redis. Rank + pick top-3.
Optimize: Move ETA prediction to GPU inference tier for L7 (DeepETA). At L6, use haversine-with-traffic-multiplier as fast approximation.
Atomic SETNX lock prevents same driver being dispatched twice. TTL 10s (long enough for driver to accept/reject).
APNS (Apple) or FCM (Google) push notification to driver's phone. External providers add significant tail latency — push delivery is 'best effort'.
Optimize: Fall back to WebSocket if driver has app open. WebSocket delivery <100ms; push notification tail is 3+ seconds.
Response to rider: 'driver assigned, ETA 4 min'. Note: this fires BEFORE driver accepts.
The rider-facing p99 is ~130ms (network + DISCO logic + Redis + response). BUT the driver push notification tail (p99 800ms, p999 3s) is what the rider actually WAITS for before seeing 'driver on the way'. **The perceived UX is bimodal**: fast dispatch acknowledgment + slower driver accept confirmation. Design apps around this — show 'searching' during push tail, not 'waiting'.
Bottleneck summary
Ride-sharing latency is DOMINATED BY CELLULAR NETWORK (100-200ms p99 per hop). Our infrastructure (H3 hex compute + Redis + Postgres) contributes <30ms typical. **The critical UX moment** is push notification delivery to phones — external APNS/FCM tail (3+ seconds p999) is what riders 'feel' when waiting for driver confirmation. Design apps for optimistic UI + explicit 'searching' states rather than polling.
Optimization tips (this architecture)
- **Persistent WebSockets over polling**: 30-second heartbeats vs on-demand connect. WebSocket ping/pong at 30s vs new HTTP request cuts overhead 5-10x.
- **Batch location updates**: Send every 8s instead of 4s. Halves network volume + battery drain. Accept 8s stale positions (dispatch decisions are still fast enough).
- **Regional DISCO clusters**: US-East, EU-Frankfurt, APAC-Singapore. Cross-region dispatch (rider in Tokyo, driver in Osaka) still works but crosses regional boundary.
- **H3 resolution tuning**: Res 9 (~150m) is good for dense cities; Res 8 (~450m) is enough for rural. Adaptive resolution based on driver density cuts Redis load.
- **Push notification fallback**: WebSocket first (if app open), APNS/FCM second. WebSocket delivery <100ms; push notification tail is 3+ seconds.
- **Optimistic UI**: Rider sees 'Searching' immediately, then 'Driver found' when push arrives. Don't block UI on push delivery — assume it works, show retry if it doesn't.
- **Acks=all only for ride commit**: Location updates use acks=1 (leader-only). Ride commit needs full durability. Balance throughput + safety per operation.
- **Kafka async for analytics**: Never block driver ACK on Kafka publish. Fire-and-forget, monitor lag separately.
Where to go next
Now that you can see where latency comes from, trace how the architecture EVOLVES to handle 10x more traffic. Or dive into the masterclass for the full ADR + business exercise + incident narrative.