Distributed Search Engine: Junior → Architect Evolution
The mandated interactive flow. Step through each rung, ask what breaks FIRST, weigh the options, and defend the chosen architecture. This is how architectural thinking is learned — not by reading a fixed design, but by tracing how it evolves under growth pressure.
Scale Evolution Timeline
Step through 6 architectural rungs, from 10K RPS to 1B RPS. At each rung, ask: what will break FIRST? Why? What options exist? Which one do we pick — and what are we accepting?
This is the reasoning cycle that separates a Junior developer ("here's an architecture") from an Architect ("here's why this architecture, why now, and what breaks next").
10K RPS — Postgres tsvector + GIN index
1. Current architecture
Where we are before growth pressure
3 apps + Postgres db.r6i.large with tsvector column + GIN index on searchable text. Simple LIKE '%query%' queries replaced with `WHERE tsv @@ plainto_tsquery(?)`. Ranked by ts_rank.
2. Growth trigger
What changed — the traffic/data force
Small catalog / knowledge base. ~1M documents. 10K search RPS. Simple keyword matching + basic ranking is enough.
3. Bottleneck — what breaks FIRST?
The component that saturates as growth arrives
None yet — Postgres full-text works
1M documents × Postgres tsvector + GIN index handles 10K RPS at 40% CPU. p99 search: 20-80ms. Ranking is basic (ts_rank_cd — not BM25 but close enough for small catalogs). No autocomplete yet, no synonyms.
Postgres CPU 40%, GIN index size 500 MB, search p99 60ms, index rebuild time 45s
4. Options — what could we do?
Alternatives an architect must consider before picking
- + Zero new infrastructure
- + Transactional consistency (writes immediately searchable)
- + SQL native + joins with other tables
- + Team already knows Postgres
- − Ceiling ~1-10M documents
- − No BM25 (weaker ranking)
- − No sharding, no vector kNN, no fuzzy matching without pain
5. Chosen
The specific decision we're making
Do nothing — Postgres full-text is right at 10K RPS + 1M documents
6. Trade-offs
What we're explicitly accepting to move forward
- Accept ceiling ~10M documents
- Accept weaker relevance (ts_rank vs BM25)
- Accept no vector kNN
- Accept GIN index maintenance during writes (~10% CPU overhead)
7. New architecture
The system after this decision — headroom for the next 5-10x
3 apps + Postgres tsvector + GIN. Total cost: $400/mo.
8. Next bottleneck — what will break at the NEXT rung?
This is the seed of the next rung
At ~100K RPS with 10M+ documents, GIN index rebuilds impractical, ranking quality complaints, no autocomplete or fuzzy. L5 shape.
What comes next in your Junior → Architect journey
You've traced 6 rungs of Distributed Search Engine evolution. Now try the same reasoning cycle on a system you don't know yet — pick from the Systems catalog and answer the same questions: current arch → growth trigger → bottleneck → options → chosen → trade-offs → new arch → next bottleneck. That is architecture.