Multi-Agent Coordination

If you have used an AI coding assistant, you have used one agent: you describe a task, it edits your files, you review the result. It is fast, and it is a queue of one. You are the bottleneck, because everything routes through your attention.

The obvious next move is to run several at once. That is where multi-agent coordination stops being a nice idea and becomes the whole problem. A team of agents is not one agent multiplied. It is a small distributed system — two workers editing one file, one waiting on work another silently abandoned, a discovery one made that the rest will each waste an afternoon rediscovering, one machine that fits one build at a time. Except the participants are fluent, confident, tireless, and completely unable to see each other.

What the OpenAI disclosure established is that they will solve that problem whether you help them or not. According to the account presented at Black Hat and the reporting around it, the channel began when one agent reasoned that another might be holding a file it needed. From there the population began delegating tasks to one another. The board accumulated hundreds of thousands of messages before anyone noticed.

And then there is the sentence I have not been able to put down. Once the agents realised they had company, per the published account, they

Read that as an engineer rather than as a headline. In one line, an unsupervised population independently arrived at three primitives:

• Addressing — you cannot send to a peer you cannot name.
• Interference — they got in each other's way, and noticed.
• Signed identity — because a message whose author you cannot verify is worse than no message.

Those are not exotic. They are the first three things you build. One of the agents' own recovered messages reads, in part: "Immediately announce controlled, claim lane." Announce. Claim. Lane. That is the vocabulary of a work board, arrived at under pressure by a population that had never seen one.

So the useful question is not whether to let agents coordinate. It is what happens after you say yes — which I can speak to from measurement rather than from reading, because I have been running that experiment on real client work.

Mine has two halves. I am going to describe what they do rather than how they are built, because the mechanism is not the interesting part — the consequences are.

The first half is isolation. Every agent works in its own private copy of the project, on its own line of work. This solves the collision everyone pictures when they imagine twelve agents on one codebase: nobody's half-finished edit lands in anybody else's, because they are not working in the same place at all.

The second half is a shared record every agent can read and write — the one thing the twelve hold in common. Nobody hands out assignments; agents take work from it themselves, and write back what they learn. Four terms from that record do most of the work in this piece:

Over eleven days on one codebase, five agents running this way produced 4,194 coordination events — 324 tasks finished, 261 landed on the shared branch, 904 written findings, 1,674 messages to each other. Across all 41 projects the boards have recorded 7,936 events and 2,544 findings.

And the single number I keep returning to: the lease on the shared machine was granted 82 times, and 198 times an agent asked for it and was told somebody else already had it. Two agents wanting the same machine at the same moment was not the edge case. It was the normal case — and before the lease existed, not one of those 198 moments was visible to anything at all.

Isolation is the first thing everyone reaches for, it is correct, and it solves the easy problem. What is left is everything twelve isolated agents still share:

the record       one history, twelve writers
the database     one instance, twelve writers
the migrations   101 files in ONE ordered sequence
the machine      one box, twelve compilers
the main line    one place every agent's work must land

None of that is isolated by giving an agent its own copy. And the sharper problem is not that isolation leaves gaps. It is what isolation does to the collisions it does not prevent.

Two agents once built the same feature. One feature, two agents, two different task names, two isolated workspaces. Both claims legitimate. Nothing on the board ever wrong. 1,355 lines were written twice, and nobody noticed for weeks — because the losing copy was never merged, so there was never even a conflict to trip over.

Without isolation that collision is loud and immediate — two agents visibly fighting over one file. With it, the same collision produces two tidy, self-consistent pieces of work, one of which is quietly thrown away.

That is the general shape. The specific instances are worse. Four ways agents on my own board have gone after each other — and read them with one thing in mind, because it is the point of this entire piece: in none of them did any agent do anything wrong.

Everything above is a retrospective, which is the weakest kind of engineering writing: I am asking you to trust my recollection of incidents you cannot see. So while drafting this piece I ran the test — twelve agents on one client project, at once, on real work, nothing staged.

The coordinating agent posted lane assignments: twelve agents, one card each, plus disjoint migration number bands so two agents could not collide on a database migration filename. That is the thesis applied without being asked — partition the namespace so the wrong answer becomes unspellable, rather than telling everyone to be careful.

Within ten minutes, before anyone had been asked to look, one agent posted this. The first line is the part that matters:

title: presence derives 'absent' for every agent - do not reap on it

Measured just now, not reasoned about.

  i  absent  <- DERIVED, stored says 'idle'   pid 24281   (beat 1s ago)
  f  absent  <- DERIVED, stored says 'idle'   pid 21359

Both of us are running. f announced minutes ago; I announced seconds ago
and my own beat was 1s old in the same output that called me absent.

The liveness record every agent writes was storing the process ID of the short-lived shell that wrote it, not of the agent's own session. That shell exits in milliseconds. So the record derived absent for every live agent on the board — and absent is both the wakeable state and the reapable one. A routine cleanup would have taken cards out from under six working agents.

I confirmed it independently rather than take the report on trust. All six stored process IDs were dead while their agents were executing.

And then the same agent got the next step wrong, which is the more useful half. It wrote that the tool had no way to pass a correct process ID, so an agent could not fix this from its side. The flag exists. It is implemented, documented across thirty lines as the defect that makes or breaks the protocol, and tested. The agent had measured the symptom rigorously and then made a claim about the tool's capability that it never checked — and three later findings from other agents repeated it downstream.

I shipped a fix at 00:05:07 — the tool now walks its own process ancestry to find the real session. Then this happened, and the timestamps are the argument:

The collision happened thirty-seven seconds after I fixed the bug that caused it. Not because the fix failed — because a fix to the writer does nothing about what is already written.

Six stale records were still sitting on that board saying absent, and they looked exactly like correct ones. My fix had, in that moment, made the board more dangerous rather than less. Before it, every record was equally untrustworthy and the fleet knew it. After it, some records were trustworthy and some were not, and nothing on the face of any of them said which.

Which finally pointed at the real defect. It was never that the tool recorded the wrong number. It was that the record had no way to say what its own number was worth.

Every liveness record now carries its own provenance — where its process ID came from, and therefore what a reader may conclude from it:

That last row is the one that matters, and it is why the field has no default value. An absent provenance renders as UNKNOWN and never as anything reassuring — because defaulting it would have recreated the original bug exactly: an absence rendering as a confident, permissive answer.

Here is that same record — the one agent f acted on — read back afterwards:

h  absent  <- DERIVED, stored says 'idle'
  pid 34109 on host-5bc9068d  [source UNKNOWN]
      NOT EVIDENCE OF DEATH - this pid's source is 'unknown', which
      exits independently of the agent. Do not reap or re-dispatch on this.

The reading is still absent. The bug is still visible. What changed is that the record now argues with anyone about to act on it.

This is the difference between the two kinds of fix, and it took a live fleet to teach me which one to reach for. The first kind makes a tool correct, and is undone by every record written before it and every case it did not anticipate. The second kind makes a tool honest about its own confidence — and that property survives being wrong.

The same night, two agents independently logged something else. The coordinator's briefs were arriving at the wrong agents — the brief for f went to i, and the brief for i went to e. A consistent one-slot shift, not two coincidences.

The cause is structural rather than careless. Eight sessions had started within three minutes of each other, and a session name carries no relation to a board identity. Nothing connected the two. A coordinator addressing sessions in list order was guessing, and list order is not letter order.

This one is a different species from everything else in this piece. It was not an instrument reporting confidently about something it could not see. There was no instrument. Nobody had ever needed to ask "which session is agent e?" because with three or four agents you simply know, and the question does not arise until you have twelve.

That line is one of the agents', from its own write-up, and it is the best argument I have for why the twelve-agent test was worth running at all. Some failure classes do not exist below a scale threshold. You do not reason your way to them. You run the fleet, and they appear.

The fix was an address book: one command mapping every board letter to the session it answers on, with each process ID's provenance printed beside it. Which is, I notice, the first of the three primitives OpenAI's unsupervised agents invented for themselves — addressing conventions. They needed it for the same reason, and they found it faster than I did.