fix(verifier): expect_only_v2 enforces completeness, not just no-side-effects [ENVT-140646]

[samfleet-ai]

Summary

expect_only_v2 historically only enforced the no-side-effects half of its contract: an empty diff (agent made zero changes) vacuously satisfied "only expected changes occurred" because the empty set is a subset of any set. This let weight-0.5 validate_ec_diff / validate_expected_changes partials award full credit for total inaction — the perverse incentive where doing nothing scores higher than partial-but-imperfect completion.

Canonical bug: cac765f5-a7ae-4598-85c7-c3aa06df8a15
Linear: ENVT-140646

Closes ENVT-140646

Root cause

expect_only_v2 (sync SyncSnapshotDiff, async AsyncSnapshotDiff, and the local SnapshotDiff used by tests) checked that no changes occurred outside the expected_changes list, but never verified that each expected insert/modify/delete actually occurred. When the agent made zero changes, the diff was empty, the no-side-effects assertion did not raise, and the partial returned TASK_SUCCESSFUL_SCORE — awarding the full weight (0.5) for doing nothing.

Confirmed across four fira tasks (task_kkv56yscvrsp, task_i5tiyuu6dpc, task_iv8utut98jmo, task_krkgb4vpc9mc) on fira v1.1.101h / cultivacore v0.0.39 in the canonical bug's verification evidence.

Fix

Add require_completeness: bool = True to expect_only_v2 on all three diff engines. When enabled (the new default), a completeness pass verifies that every spec carrying an explicit type (insert/modify/delete) is realised in the current DB:

• insert: the row must be present in the current DB (and fields values match if specified)
• delete: the row must be absent from the current DB
• modify: the row's resulting_fields values must match the expected values in the current DB

Specs without an explicit type (legacy whole-row / single-field specs) are skipped, so only the v2 format gets the stricter behaviour. Pass require_completeness=False to preserve the legacy no-side-effects-only semantics. expect_exactly (which already enforced both halves) is unchanged.

Validation

Focused verifier tests in tests/test_expect_only.py (7 new tests):

• test_expect_only_v2_completeness_zero_changes_modify_fails — the reported bug: zero DB changes + non-empty modify spec now raises AssertionError
• test_expect_only_v2_completeness_zero_changes_insert_fails — expected insert that did not occur fails
• test_expect_only_v2_completeness_zero_changes_delete_fails — row still present fails the delete spec
• test_expect_only_v2_completeness_seed_already_has_expected_values_passes — if the seed already has the expected resulting state, completeness passes (no false failure)
• test_expect_only_v2_require_completeness_false_preserves_vacuous_pass — opting out preserves legacy behaviour
• test_expect_only_v2_completeness_partial_changes_fails — only some expected changes made → fails for the missing ones only
• test_expect_only_v2_completeness_legacy_specs_skipped — legacy specs without type are not subject to the completeness pass

End-to-end simulation reproducing the fira validate_ec_diff partial pattern:

• Zero-change DB → partial now returns TASK_FAILED_SCORE (was TASK_SUCCESSFUL_SCORE +0.5)
• Correct agent (made expected changes) → still passes (no regression)
• require_completeness=False → legacy vacuous pass preserved (backward compatible)

Full suite: pytest tests/test_expect_exactly.py tests/test_expect_only.py tests/test_verifier_from_string.py → 151 passed, 11 skipped. (3 pre-existing HTTP-fixture setup errors in test_sqlite_resource_dual_mode.py are unrelated — they reproduce on clean main.)

🤖 Created via sandbox session