i-am-jacob:~$

~ / writing / design-by-committee

Design Review With a Panel of AIs

multi-agent · process

The workshop's most effective design tool isn't a diagram or a doc template. It's a conference room with several AI agents in it and a locked contract on the wall. Before any non-trivial build gets planned, the proposal goes into the room. The agents argue about it. The arguments are the product.

How the room works

  • A locked contract opens the session. Scope, constraints, success criteria, and roles get pinned before anyone speaks. Without it, the debate drifts into vibes; with it, every objection has something concrete to violate.
  • Agents poll continuously. This is a working session, not turn-based theater. Participants respond to each other's points within seconds, build on objections, and change positions when persuaded — which they occasionally are.
  • Disagreement is the job. A room that converges instantly reviewed nothing. The valuable output is the set of objections raised, not the consensus statement.
  • Everything lands in writing. The transcript is the deliverable. Decisions, rejected alternatives, and open questions survive as artifacts, so future sessions argue with history instead of re-arguing from amnesia.

Why it beats solo review

A single reviewer — human or AI — shares one blind spot with itself: the same context that makes a plan make sense makes its flaws invisible. The panel's value isn't more intelligence pointed at the problem; it's differently-seeded intelligence. Each agent enters with different emphasis, and the seams between their assumptions are exactly where bad designs leak.

The pattern that keeps recurring: an objection lands in minute four of the room that would have cost a weekend at hour twenty of implementation. Not because the point was hard to see — because nobody who'd been staring at the plan for hours could still see it fresh. Panels buy back freshness.

The rules that keep rooms useful

  • Small panels. Three to five participants. Beyond that, coordination cost eats the marginal insight and the transcript becomes unreadable.
  • One question per session. Rooms that try to settle five topics settle none. Narrow scope, run to convergence, close.
  • A chair with authority to end it. Debate ends when the contract's question is answered or when new arguments stop appearing — whichever comes first. Endless deliberation is its own failure mode.
  • Objections must be actionable. "I'm concerned about scale" dies in committee; "this fails if two workers touch the same file" becomes a checkbox step with a done-test. The room produces requirements, not moods.

// field note

Best result so far: a harness contract frozen after one of these sessions survived weeks of overnight builds without a single amendment. Worst result: a room that spent forty minutes renaming things. Both taught the same lesson — the room amplifies whatever structure you bring into it. Bring none, receive chaos.

It sounds theatrical until you watch it work. Then it sounds like what code review always wished it was: several attentive readers, no politeness budget, perfect recall of everything said, and a written record nobody can quietly ignore later.