Distributed Job Scheduler: Junior → Architect Evolution
The mandated interactive flow. Step through each rung, ask what breaks FIRST, weigh the options, and defend the chosen architecture. This is how architectural thinking is learned — not by reading a fixed design, but by tracing how it evolves under growth pressure.
Scale Evolution Timeline
Step through 6 architectural rungs, from 10K RPS to 1B RPS. At each rung, ask: what will break FIRST? Why? What options exist? Which one do we pick — and what are we accepting?
This is the reasoning cycle that separates a Junior developer ("here's an architecture") from an Architect ("here's why this architecture, why now, and what breaks next").
10K jobs/day — cron on single server
1. Current architecture
Where we are before growth pressure
1 dedicated server running Linux cron. Scripts + timing in /etc/crontab. Log to files, rotate nightly.
2. Growth trigger
What changed — the traffic/data force
Early team. 10 cron jobs: nightly DB backup, hourly log rotation, daily reports.
3. Bottleneck — what breaks FIRST?
The component that saturates as growth arrives
None yet — Linux cron is fine
Linux cron battle-tested. Handles 100+ jobs on single server easily. Logs to /var/log/cron.
Jobs succeed 99%+, log volume 1 MB/day, no missed runs
4. Options — what could we do?
Alternatives an architect must consider before picking
- + Zero infrastructure
- + Team knows Linux
- + Battle-tested
- − Single server = SPOF
- − No visibility (must SSH to check)
- − Missed schedules if server restarts near cron time
5. Chosen
The specific decision we're making
Do nothing — cron is genuinely right at MVP
6. Trade-offs
What we're explicitly accepting to move forward
- Accept single-server SPOF
- Accept SSH-only visibility
- Accept missed schedules on restart
7. New architecture
The system after this decision — headroom for the next 5-10x
1 EC2 + Linux cron. Total cost: $50/mo.
8. Next bottleneck — what will break at the NEXT rung?
This is the seed of the next rung
At ~100 jobs/day with mission-critical schedules (nightly reports, quarterly billing), SPOF is unacceptable. Also 'did the job run today?' requires SSH check. Leader-elected scheduler + monitoring. L5.
What comes next in your Junior → Architect journey
You've traced 6 rungs of Distributed Job Scheduler evolution. Now try the same reasoning cycle on a system you don't know yet — pick from the Systems catalog and answer the same questions: current arch → growth trigger → bottleneck → options → chosen → trade-offs → new arch → next bottleneck. That is architecture.