When a second agent earns its keep
Most multi-agent systems are one agent wearing a hat and paying for two. A test for whether the second one is real, and what to do instead when it isn't.
Every architecture diagram I am sent has at least four boxes labelled agent. Usually a Planner, a Researcher, a Writer and a Critic, arranged around a Supervisor, with arrows going everywhere.
Then you open the code and all four are the same model, with the same tools, called with a different system prompt. That isn't a multi-agent system. It's one agent, four times the latency, four times the token bill, and four times the surface area for something to go wrong in a way you can't reproduce.
I'm not against the pattern — I build systems with several agents in them and they're better for it. But the second agent has to earn its place, and there's a straightforward test for whether it has.
The test
Does the second agent have a different contract?
Not a different personality. A different contract. Concretely, at least one of these has to be true:
- Different tools. It can reach something the first one cannot, or — more usefully — it cannot reach something the first one can.
- Different output schema. It is validated against a different shape, so a malformed answer fails in a different place.
- Different inputs. It genuinely sees less. A critic that reviews the output without seeing the reasoning that produced it is doing real work. One that sees the whole transcript will mostly agree with itself.
- Different failure handling. A timeout or a refusal from this step means something different to the system than a timeout from that one.
If none of those is true, you have written a prompt, not an agent, and you should chain it inside one call.
The permission boundary is the best reason
The strongest case for splitting is almost never capability. It's authority.
In a system I built for a fund management platform, a drafting agent composes an investment risk narrative and a separate step commits anything. The drafter has read tools and no write tools — not by convention, by wiring. It could not write to the database if it decided to, because the tool server it talks to does not expose an operation that writes.
That's worth the extra hop on its own, and it survives contact with reality in a way that "the prompt says not to" does not. When somebody asks what stops it doing something stupid, "there is no function it can call to do that" is an answer. "We told it not to" isn't.
Same shape in the small assistant behind a local business site: the agent that reads an enquiry and prices it can look up the rate card, and the agent that drafts a follow-up message cannot send anything at all. Sending is a button on the owner's phone.
Route with code where you can
The other thing worth being deliberate about is who decides where the conversation goes next.
An LLM router is a model call whose entire job is to pick a branch. It costs a round trip, it costs tokens, and — the part people underestimate — it produces a class of bug you cannot write a unit test for. The same input picks a different branch on Tuesday and you have nothing to assert on.
For a closed set of intents, this is a classifier, and often not even that:
def route(state: State) -> str:
if state.quote_requested and state.slots_filled:
return "quote"
if state.missing_slots:
return "ask"
if state.intent == "complaint":
return "human"
return "answer"Boring, free, instant, and testable. Keep the model for the step where language is the actual problem — reading a messy WhatsApp message and pulling structure out of it. Use it to decide what the user said, not what the program does next.
Where an LLM router genuinely helps is an open-ended space you can't enumerate. Most business workflows are not that. They have six paths and someone can name all six.
State, not transcript
The single change that made these systems maintainable for me was carrying typed state between steps instead of an ever-growing message list.
class ConversationState(TypedDict):
intent: Intent | None
slots: Slots # occasion, date, area, guests, budget
missing: list[str]
quote: Quote | None
approved: bool
trace: list[Step]Three things fall out of this that you don't get from a transcript.
You can assert on it. assert state["quote"].subtotal == 10397 is a test. "The reply mentioned about ten thousand rupees" is not.
You can replay from the middle. When a run goes wrong at step five, you load the state as it was at step four and run forward. With a message list you re-run everything from the top and hope for the same weather.
You can see the routing. The reason a run took a path is a value you can print, not an inference from a paragraph the model wrote about its own reasoning — which, to be blunt, is a story generated after the fact and is not evidence of anything.
The bill nobody models
A supervisor loop multiplies. Five agents, each averaging three turns, each turn carrying the accumulated context, and you are at fifteen model calls with a context that grows on every one of them. I have seen a "simple" agent graph cost forty times what the equivalent single call cost, for an answer that was not forty times better, or better at all.
Before adding an agent, it's worth writing down what the extra call is buying — a permission boundary, a genuinely independent check, work that can run in parallel — and if the honest answer is "structure in the diagram", delete it.
Where I do fan out
To be fair to the pattern, three cases where more agents are clearly right:
- Parallel work over independent items. Ten documents to summarise is ten calls that don't need to talk to each other. That's not a multi-agent system, it's a
gather, and it's the cheapest win in the category. - A real adversary. A checker that receives only the final output plus the source material, with no sight of how it was produced, and whose contract is to return a list of specific defects. Give it the reasoning and it will nod along.
- A hard authority split, as above. Read and write should not be the same identity.
What I'd do first
If you have a four-box diagram and you're not sure it's real, collapse it. Write the whole thing as one call with the tools attached and the steps in the prompt, and measure. Then split out the one step where you can name what a separate contract buys you.
It's much easier to add the second agent later than to work out which three of your four are decoration once they're all in production.