fix(indexer): keep same-ledger REMOVE so deletes are never lost

[JiahuiWho]

What

Fix a bug where the indexer could permanently keep a stale balance row when a trustline (or account / SAC / liquidity-pool balance) was deleted and re-created within a single ledger. The buffer now keeps the last change per key instead of cancelling out a same-ledger create+delete pair, so a trailing delete is always applied.

Changes are:

1. Removed the pushWithTombstone function and its five *IsNoopRemove helpers, the logic that cancelled a same-ledger create+delete pair down to no write.
2. Replaced it with pushHighestOrder, which simply keeps whichever change has the highest operation order per key, i.e. the last operation to touch that key.
3. A trailing delete is now emitted as a real delete instead of being cancelled.

Why

The indexer collapses each ledger's balance changes into one write per account+asset, and dropped a same-ledger "create then delete" pair as a no-op.

That was only safe if the row didn't already exist in the database, but it never checked. So a transaction that in one ledger deletes an existing trustline and then re-creates and re-deletes it (delete → create → delete) wrote nothing: the re-create overwrote the pending delete, and the final create+delete cancelled out. The old row was never removed.

Known limitations

Rows already corrupted on main are not repaired by this PR, a one-off backfill/cleanup is needed separately.
Also the live-ingest-processor-perf and live-ingest-persist-path branches touch the same file and will re-introduce the bug on merge unless rebase or same fix is applied there.

Issue that this PR addresses

Reported via HackerOne #3941742.

Checklist

PR Structure

• It is not possible to break this PR down into smaller PRs.
• This PR does not mix refactoring changes with feature changes.
• This PR's title starts with name of package that is most changed in the PR, or all if the changes are broad or impact many packages.

Thoroughness

• This PR adds tests for the new functionality or fixes.
• All updated queries have been tested (refer to this check if the data set returned by the updated query is expected to be same as the original one).

Release

• This is not a breaking change.
• This is ready to be tested in development.
• The new functionality is gated with a feature flag if this is not ready for production.

[Copilot]

Pull request overview

Fixes stale indexed balance rows by preserving the highest-order change, including trailing deletes.

Changes:

• Replaces tombstone-based netting with highest-order deduplication.
• Preserves same-ledger trailing removals.
• Updates regression tests and documentation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
internal/indexer/indexer.go	Clarifies deterministic operation ordering.
internal/indexer/indexer_test.go	Updates operation-order regression expectations.
internal/indexer/indexer_buffer.go	Implements highest-order change retention.
internal/indexer/indexer_buffer_test.go	Covers retained deletes and ordering behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.