fix(engine)!: charge finalization fees against the state actually persisted

[sdbondi]

Stack: #2416 ← this PR. Based on #2416; review the top commit only. Merge #2416 first.

Problem

Every finalization charge is computed against the live working state. But a transaction that cannot pay for its main intent commits only its fee intent, and that diff comes from the fee checkpoint — a snapshot taken when the fee instructions ended.

So Storage, SubstateCreate and TemplatePublish were priced over substates that are then thrown away, and ExhaustBurn inherited the inflation since it is taken over the running total. This predates #2416; pricing the receipt just made it visible, because the receipt bound scales with the discarded state's substate count and events.

Measured on fail_partial_paid_fees: 1019 µT of storage charged against a receipt and substate set that occupy 444 µT.

Why the obvious fix doesn't work

Charging only at the end cannot fix this on its own. The charge has to be in place before the paid-in-full check in select_finalized_state, or nothing gates a transaction that commits state it never paid for. But the charge decides the outcome, the outcome picks the state, and the state determines the charge — a cycle.

Charge twice

• on_before_finalize runs against the live state, as before. Its only job is to gate the commit/reject decision. Pricing what would be persisted is the right question to gate on: to commit that state you must be able to pay for it.
• on_before_persist (new hook) runs once the state has been chosen and before its fees are settled, replacing those charges with the cost of what is really written.

On a commit the second pass sees the same state and changes nothing. On a fee-intent commit it sees only what the fee intent touched.

For the replacement to be sound the finalization charges are now assigned rather than accumulated — FeeBreakdown::set — so a second pass over a different state replaces the first instead of doubling it. ExhaustBurn is zeroed before its total is taken, so it never compounds across passes.

Structure

StateTracker::finalize is split into select_finalized_state (chooses the working state or the fee checkpoint, and carries the live fee state onto it) and finalize (settles fees, builds the diff and receipt), so the runtime can drive its modules in between. The two near-identical commit and fee-intent-commit tails collapse into one. WorkingState becomes pub(crate) and gains count_newly_created_substates, so the whole computation can run against a detached state.

Consequence worth reviewing

A transaction rejected for insufficient fees now gets refunded the storage it was provisionally charged for state that was never written. Execution charges (WasmExecution, NativeExecution, RuntimeCall, …) are untouched — the work was done and is still paid for — so this is not a spam vector, but it is a real change in what a failed transaction costs.

Testing

New regression test a_fee_intent_commit_is_not_charged_for_the_state_it_abandons runs the same transaction at two fee levels and asserts the rejected run's storage charge is less than half the committed run's. With on_before_persist stubbed out it fails at 1769 vs 1772 — the two are within three bytes, which is exactly the bug.

tari_engine, engine_types, transaction, ootle_sdk_core, storage and consensus_tests all pass with no other test changes.

Follow-up

Storage is still charged in one tally at finalization, so a transaction that cannot afford its state only finds out after doing all the work — and checkpoint_fee_intent's paid-in-full check still sees no storage charge at all, since none has been computed by then. Charging incrementally during execution would close both. It composes cleanly with this PR: with the second pass authoritative, the running charge only has to be an over-estimate.

[sdbondi]

---

Update: closing the fee-intent commit hole

Charging categories earlier turned out not to be a workable defence — see the discussion on #2418. Each charge moved before checkpoint_fee_intent leaves the others behind, and the receipt is not a substate during execution at all, so it can never be metered that way. A fee intent could therefore still write state, cover only its execution, and have that state committed with the rest of the bill unpaid.

This PR now decides at the point where the answer is finally known. on_before_persist has just recomputed the charges over the exact state being persisted, so the payment is tested against them there: if it cannot cover committing, nothing commits. That is complete by construction — it tests the final total, so no charge added later can escape it.

Reproduced against a local swarm before the fix: claiming testnet funds with max_fee: 1000 against 2548 µT of charges left the account funded (999,999,000 µT, is_confirmed_on_chain: true) while the transaction reported Insufficient fees paid.

Whatever rejected the main intent stays the reason reported; the shortfall only decides that not even the fee intent survives it.

Test fallout, all of it the intended semantic change — underfunded fee intents that used to commit now reject:

• an_unaffordable_fee_intent_commits_nothing (new) — pins it, including that nothing is charged and the account is not funded.
• fail_pay_less_fees_than_fee_transaction — now expects a full reject, matching its name.
• fail_partial_paid_fees, dangling_bucket_pay_fees, a_fee_intent_commit_is_not_charged_for_the_state_it_abandons — fee raised so the fee intent can afford its own commit, which is what makes a fee-only accept reachable at all.
• fail_partial_paid_fees also now asserts the payer is charged only for what the fee intent persisted and refunded the rest — the invariant this PR introduces.