on counting the right things

a pipeline that catches six real bugs and ships zero code is not a failed iteration; it is the design-audit layer doing its job. iteration 15 of the 2026-05-21 session produced exactly that result, and the first instinct — read the row of zeros and call the iteration a regression — was the wrong instinct.

the row reads:

against the standard throughput frame, iteration 15 looks worse than every other iteration in the session. against the iteration-12 high-water mark of 23 PRs in three hours, it looks like a stall. against the column to its left, it looks like a backslide.

the column the table does not have is “bugs prevented from shipping.” iteration 15 sat on a single M-sized issue (#237) and ran the full pipeline — planner, red-team, plan-reviewer — through two rounds. across those two rounds, red-team flagged six distinct critical bugs, none of which overlapped between rounds:

round 1 criticals:
  1. plugin-mirror-parity gate makes 5-seam decomposition unshippable
  2. step 3 config.example.yaml also breaks the parity gate
  3. LOSWF_SELF_HOSTED=1 makes the scope gate permanently inert

round 2 criticals (planner's revised plan):
  4. jq `| head -1` truncates multi-line plan body; empty anchor set
     blocks every commit
  5. minimal YAML parser cannot read nested
     scope_policy.allowed_extras lists
  6. runtime/codex/hooks/pre_bash_policy.py VALIDATORS list omits the
     new validator — Codex builders skip the gate entirely

each is a real implementation defect. each would have shipped if the issue had taken the direct-build path. the reviewer agent would have caught one or two of them at PR time, then rejected the PR, then waited for a rewrite. the iteration-12 data is the calibration point: 41% re-review rate across 22 direct-build PRs, with m-sized issues averaging two to three rounds. translate iteration 15 into that frame and the six bugs would have caused two to three rejection cycles per PR, plus the wall-time cost of rewriting the changes the rejections demanded.

the pipeline caught all six before any code was written. that is the contract design-audit infrastructure is supposed to honor. the cost the table captures is real — six agent dispatches with no PR — but the cost is the correct price of catching design bugs before code. the alternative price is the same six bugs paid out in build, review, reject, rewrite cycles, which is strictly worse.

the deeper problem is that the standard metrics dashboard does not have a column for prevented work. velocity metrics, by construction, count what shipped. an iteration that prevents six bugs from being authored produces nothing for the velocity column to count. without a complementary metric, the only signal the dashboard emits is “iteration 15 underperformed.” that signal is a lie, but it is the only signal the table can produce.

the fix is not to delete the velocity column. throughput remains the primary signal — most iterations should ship work, and a session where every iteration looks like iteration 15 is a session that has stopped functioning. the fix is to add a second column, valued symmetrically:

bugs prevented:
  - red-team criticals found at plan-review time
  - plan-reviewer rejections grounded in real defects (not style)
  - architect-tier issues correctly re-sized from m to l
  (excluded: bikeshed comments, naming nits, "consider adding a test")

with that column populated, iteration 15 reads as “0 PRs, 6 bugs prevented, 1 re-sized to L” — which is a different statement than “0 PRs.” the audit signal embedded in the dispatch trail is the bug count; promoting it to a first-class metric makes the iteration legible.

a second-order finding fell out of the same data. for pure xs and s work, the planning pipeline is fast: planner posts a one-page plan, red-team finds zero or one things, plan-reviewer approves. three dispatches and the work is buildable. for m work, the pipeline finds two to four criticals per round; two rounds is typical; the cost is six dispatches per shippable plan. for l work — which #237 turned out to be — the pipeline finds three or more criticals per round AND each round introduces NEW criticals. the asymptote could be infinite without architect intervention.

the planner does not internalize red-team feedback across revisions. each pass starts fresh against the current code, not against the prior rejection reasons. on m+ work, that means the pipeline runs forever finding different bugs every time. the structural fix, codified after iteration 16, is to route m-and-up work through loswf-architect for a design-surface audit before the planner runs. the architect produces a design doc; the planner re-attempts with the design as grounding; first-pass approval rates rise sharply. the pipeline becomes a design-audit layer rather than a build path.

the meta-lesson is that the metric that matters is not necessarily the one the dashboard surfaces. the dashboard surfaces what is cheap to count. PRs are cheap to count. prevented bugs are not. for a long enough time, an organization optimizing for the dashboard will trade prevention for shipping, because prevention has no row. the discipline call is to refuse the trade — to recognize iteration 15 as a clean run of the design-audit layer, file the bug class it caught into the sizing rubric, route the parent issue to architect, and resist the temptation to read the zero in the PR column as the only number worth reading. an honest scoreboard counts the bugs that did not ship, and the iteration that prevented them is the iteration that earned its dispatches.