feat(metrics): per-agent toolCalls in agents[], from the run's own walk

[thedavidmeister]

Every agents[] entry in a stage: "final" metrics row now carries toolCalls
beside its usd / messages / cacheRead / cacheWrite, so "did workers make
fewer calls after change X" is answerable from metrics/runs.jsonl alone.

Before this, the run-level toolCalls was a single total and the only way to
split it per worker was jq over parent_tool_use_id in the raw trace — the
artifact that rotates, against a row that is kept forever. Getting the "23
poll-shaped calls" figure in #330 needed a 20 MB trace that only still existed
because the run was 21 hours old.

What it does

StartupProbe — the walk that already produces the run-level toolCalls — now
counts each call to its actor as well as to the run, keyed by parent_tool_use_id
(__main__ for the main loop), which is exactly how token_attribution already
groups spend. final_record reads that map onto the agents[] rows it emits.

Three things make it hard to drift:

• One writer. StartupProbe::record_call is the only place either counter
  moves, so the partition IS the total by construction rather than by two += 1s
  that a future guard could be added to one of.
• One key. owner_key is the single derivation of "whose event is this",
  used by both the spend walk and the call walk. If they ever disagreed, a row's
  per-agent usd and per-agent toolCalls would describe different populations
  while both looked fine.
• The backfill recounts. backfill-metrics rebuilds agents wholesale, so
  a backfill that did not recount would strip the field off rows that had it.

startupToolCalls per agent: left out, deliberately

The issue asks for a decision on the record. It is not well-defined per
worker, on two counts the code settles:

1. At run level it counts calls before firstMutationIndex, and
   is_mutation_tool recognises the run's org mutations — gh pr create,
   git push, git commit, or the vetter's record_verdict. A rework worker
   that edits a diff and hands it back to the main loop never issues one, so its
   per-worker analogue would read "every call was startup" for a worker that did
   nothing but work.
2. On the runs measured below, the workers that do push are a minority of the
   rows. So the field would mean "orientation overhead" on some rows and "this
   worker never got productive" on others, decided by which row you are looking
   at — the number whose meaning differs by row that agents[] carries no toolCalls, so per-worker call counts need the trace the sweep deletes #330 says to leave out.

One honest field, per the issue.

Measured on real traces

The two post-change producer runs from the issue's table, run through the new
binary, against an independent jq grouping of the same traces:

run	run toolCalls	sum of agents[].toolCalls	jq oracle
20260817T110404Z	376	376	376 (15 main + 118/92/82/61/8)
20260817T050103Z	237	237	237 (19 main + 63/53/52/27/23)

Per-worker, per-owner counts match the jq grouping exactly, not just the total.
The question the issue could not answer is now a field read: mean tool calls per
rework worker was 72.2 on 110404Z and 42.7 on 050103Z.

On the base binary the same command emits "toolCalls": null for every one of
those six agent rows.

Closes #330

QA

• Discriminating tests: usage_probe_tests::per_agent_tool_calls_partition_the_runs_own_total (per-label counts AND the sum identity), usage_probe_tests::spend_and_calls_are_keyed_to_the_same_actor (the two walks agree on the owner for null / absent / "" / a real id), usage_probe_tests::the_backfill_recounts_per_agent_tool_calls — each fails on base by construction: RunMetrics::tool_calls_by_owner, owner_key and agent_row's second parameter do not exist there, and the observable base behaviour is agents[].toolCalls == null on every row of a real trace (pr-review-report run-metrics runs/20260817T110404Z.jsonl at 307a598, output in the body above). Baseline suite green at 307a598: 1412 passed, 0 failed; with this change 1415 passed, 0 failed.
• Mutations applied: StartupProbe::record_call → drop the per-owner increment, keep the run total → kills all three; final_record agents map → look every agent up on MAIN_LOOP_OWNER (total preserved, partition destroyed) → kills per_agent_tool_calls_partition_the_runs_own_total; token_attribution → re-inline its own owner derivation with a different default instead of owner_key → kills spend_and_calls_are_keyed_to_the_same_actor (+ 3 pre-existing label tests); agent_row → drop the toolCalls field → kills the_backfill_recounts_per_agent_tool_calls; backfill_row → pass 0 instead of the recount → kills the_backfill_recounts_per_agent_tool_calls. 5 applied, 5 killed, 0 survivors.
• Oracle: the expected per-agent counts come from jq over parent_tool_use_id on two real traces (~/issue-pr-cron/runs/20260817T110404Z.jsonl, 20260817T050103Z.jsonl) — the same hand method agents[] carries no toolCalls, so per-worker call counts need the trace the sweep deletes #330 says is the only way to get the number today, run independently of this code; its totals also match the run toolCalls column of the issue's own table (376, 237). The unit-test fixtures' counts are hand-derived from the events written into them (2 dispatches + 1 main + 5 A + 2 B = 10), not read back from the implementation.
• Category check: issue asks for (a) toolCalls on every agents[] entry of a stage: "final" row, (b) a test pinning that the per-agent counts account for the run-level toolCalls, (c) a decision on per-agent startupToolCalls. Covered a, b, and c (decided against, with the reason on the record above). Closes because the field, the identity test and the ruling are the whole ask.

[coderabbitai]

Caution

Review failed

An error occurred during the review process. Please try again later.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thedavidmeister]

Merged origin/main after #279 landed. One file conflicted, pr-review-report-rs/src/main.rs, three hunks — all the same collision: this PR adds a tool_calls parameter to agent_row, main adds a touches parameter, and main additionally rerouted the live end-of-run path to call agent_row where this branch still built the row with an inline json!.

Resolved by taking both parameters and keeping main’s reroute, which is what agent_row’s own doc comment asks for — “one constructor, so the end-of-run record and the backfill cannot drift into describing a task differently.” Building the live row inline here would have reintroduced exactly that drift, and would have dropped touched/tokens from live rows while backfilled rows carried them.

Kept this PR’s comment on why there is deliberately no per-agent startupToolCalls — that reasoning is not recorded anywhere else.

One test then failed to compile: it called final_record with the pre-merge 10-argument signature. Supplied the empty TouchBlock the neighbouring tests use.

clippy -D warnings clean, 1579 tests pass.