AI Systems Architecture — Mastery2 / 9
Single Agent vs. Multi-Agent — Choosing a Topology
Multi-agent is fashionable and usually premature. Here is how to decide honestly — and why most products should start with one well-equipped agent.
"Multi-agent" sounds advanced, so teams reach for it early. Usually that's a mistake. A single agent with good tools is simpler, cheaper, and easier to debug — start there and earn your way to more.
When one agent wins
If the task is mostly sequential and fits in one context, one agent with the right tools beats a swarm. Fewer moving parts, one trace to debug, no coordination overhead. Most products live here longer than they think.
The real reasons to go multi-agent
- Parallelism — genuinely independent subtasks (N files, N sources) run concurrently for wall-clock wins.
- Context isolation — a heavy, read-intensive subtask shouldn't pollute the main agent's context.
- Specialization — a sharply-prompted reviewer or researcher behaves more consistently than one generalist juggling roles.
If none of these apply, multi-agent is just overhead wearing a trend.
The coordination tax
Decided you need coordination? Next: the orchestration patterns — pipelines, routers, and swarms.
Series — AI Systems Architecture — Mastery
- Part 01Architecting AI Products — First PrinciplesAI systems fail differently from normal software: they're non-deterministic, costly per call, and hard to test. The architecture has to account for all three.
- Part 02Single Agent vs. Multi-Agent — Choosing a Topology — you are hereMulti-agent is fashionable and usually premature. Here is how to decide honestly — and why most products should start with one well-equipped agent.
- Part 03Orchestration Patterns — Pipelines, Routers, SwarmsOnce you have multiple steps or agents, how they're wired together decides cost, latency and reliability. Four patterns cover almost everything.
- Part 04Context & Memory ArchitectureThe context window is your most expensive, most contested resource. What you put in it — and what you remember between calls — is an architectural decision.
- Part 05Evaluation Pipelines as InfrastructureIn AI systems, evaluation is not QA you do at the end — it's infrastructure you build first. Without it, every change is a prayer.
- Part 06Cost Engineering — Token Budgets That HoldAn AI feature that delights at 100 users can bankrupt you at 100,000. Cost is an architectural constraint, designed in — not discovered on the invoice.
- Part 07Latency & Throughput at ScaleInference is slow and bursty. Streaming, parallelism, and the async boundary are what keep an AI product feeling fast under real load.
- Part 08Reliability — Retries, Fallbacks, GuardrailsModels return malformed output, providers go down, and outputs drift. A reliable AI system expects all three and keeps working anyway.
- Part 09The Reference Architecture in ProductionTopology, orchestration, memory, eval, cost, latency and reliability — composed into one blueprint for an AI system that survives real users.