multi-agent-system-coordination machine learning

A multi-agent system splits a task across several LLM instances rather than asking one model to do everything, and the reason this helps is more specific than "more compute": a single agent has no built-in mechanism to catch its own mistakes, while a second agent with a different role — critic, verifier, opposing debater — can catch an error the first agent had no way to see in its own output. This is a genuinely different problem from the orchestrator-workers pattern of splitting one task into independent sub-tasks; here the agents are examining and correcting each other's reasoning, not just dividing labor.

Multi-agent debate reduces factual errors by making two agents argue opposing positions and forcing disagreement into the open. Du and colleagues (2023) showed that having multiple language model instances propose answers, see each other's reasoning, and then revise their own answers across several rounds — genuine debate rather than a single model just sampling multiple times — measurably improved factual accuracy and reasoning correctness compared to a single model's first answer. The mechanism is specifically about surfacing disagreement: when two agents reach different conclusions from the same question, that divergence is itself useful information a single model talking to itself would never generate, since a single model asked to double-check its own answer tends to simply agree with what it already said.

Error propagation across agents is a different failure mode from a single agent's own hallucination, because downstream agents have no way to distinguish a confidently wrong upstream claim from a correct one. In a pipeline where a research agent's output feeds a writing agent, which feeds a review agent, a hallucinated fact introduced at the research stage looks exactly like any other claim by the time it reaches the writing agent — there's no flag attached to it marking it as uncertain, and the writing agent has no independent way to verify it without redoing the research itself. This is why multi-agent systems built for reliability specifically include verification agents whose only job is fact-checking against a source, rather than assuming that chaining more capable agents together automatically produces a more reliable result — a chain of agents, each individually accurate most of the time, can still propagate one early error through every subsequent stage unchanged.

Coordination between agents happens through one of two architectural patterns: direct message-passing, or a shared blackboard both agents read and write to. Direct message-passing — AutoGen's conversational pattern is a well-known example — has agents send structured messages to each other explicitly, which keeps each exchange traceable but means the communication topology has to be designed in advance: which agent is allowed to message which. The blackboard pattern, a much older idea from classical AI systems like the 1970s Hearsay-II speech recognition system, instead has every agent read from and write to one shared state that all agents can see, letting agents react to whatever's currently on the blackboard without a predetermined conversation structure — more flexible for coordination patterns that aren't known in advance, at the cost of being harder to trace exactly which agent's contribution led to a given outcome, since nothing records who specifically is "talking to" whom.

Redundant work and deadlock are coordination failures specific to having multiple independent decision-makers, not failures either agent individually made a mistake causing. Two agents working from the same shared goal but without visibility into each other's current progress can duplicate effort — both independently deciding to research the same sub-topic — or worse, reach a standstill where each is waiting on the other to act first, neither one wrong on its own terms but the system as a whole stuck. This is the reason multi-agent frameworks invest specifically in termination conditions and explicit role boundaries rather than trusting agents to self-organize: a well-specified division of responsibility prevents the coordination failure before it happens, rather than relying on any individual agent recognizing and resolving it mid-task.

Read a multi-agent system as trading a single point of failure for a coordination problem: one capable agent has no way to catch its own errors, but several agents checking each other introduce a new category of failure — error propagation, redundant work, deadlock — that a single agent structurally cannot have, because there's no second decision-maker to disagree with, duplicate, or get stuck waiting on.