Skip to content

feat: improve wallet discovery and Yano mempool-aware sends - #24

Merged
satran004 merged 14 commits into
mainfrom
feat/119-wallet-indexes
Oct 5, 2026
Merged

satran004 merged 14 commits into
mainfrom
feat/119-wallet-indexes

Conversation

@satran004

@satran004 satran004 commented Sep 6, 2026 •

Copy link
Copy Markdown
Member

Restored Shelley wallets can miss funds when receive/change discovery stops at spent addresses, and subsequent sends can reuse confirmed inputs already claimed by a pending transaction. This PR improves address discovery and history recovery, and uses Yano's mempool-aware UTxO view for transaction building.

Changes

  • Scan receive and change chains using first-seen address history; persist credential-filter transaction scans and recover canonical checkpoints after reorgs. Include withdrawable rewards in the dashboard total.
  • Request include_mempool=true for Yano address/payment-credential selection listings and individual output resolution. Pending-spent inputs are excluded and pending change is eligible for ADA and native-asset sends, including hardware builders.
  • Persist local submitted-input reservations and remember the exact input/owner reported by a Yano conflict. Discard conflicting drafts and offer Rebuild & review without automatically resubmitting a stale signed transaction.
  • Preserve confirmed balance/history and existing CIP-30 UTxO queries. Yaci DevKit keeps its existing selection/submission backend and address-transaction history, without Yano scan or mempool requests.
  • Show the year in transaction history and default Yaci DevKit connections to port 8080.
  • CIP-30 multi-address signing (ADR-047). signTx/signData no longer sign only with receive index 0. A bounded search over the unlocked HD account finds the keys a request actually needs: session-known paths first, then receive/change 0–29, and 0–49 only after an explicit user extension. Targets come from required_signers and resolved spending/collateral inputs only. The approval prompt lists the matched paths and any unmatched hashes; a one-use ticket binds consent to the exact bytes, session and backend. Hardware signing is unchanged.
  • Receive address details. Each receive address can show its payment and stake verification keys and key hashes with derivation paths (public material only). Viewing one also makes that path known to the signer search.
  • Wallet scan feedback. The History screen reports a scan in flight instead of sitting empty: "Loading history…" immediately, then Scanning the chain for your transactions — block X of Y · N%, read from the ready/progress records the scan already streams. Progress is published unsynchronised (the scan holds the instance lock for its whole run) and cleared on every exit, so a failed scan leaves no frozen percentage; the poll runs on the common pool, never the backend executor the scan occupies. A node with no scan route and no local record now says "History not found" rather than "No transactions yet"; where local records exist they still show under the "Local history only" chip (ADR-043).
  • Yano 0.1.0-pre16 and managed-node sizing. The node pin moves to pre16, which serves the wallet scan index. The managed node is spawned directly rather than through yano.sh, so it now gets that script's sizing itself: quarkus.profile=<network>,<sizing>,wallet (medium by default; wallet stays last so its index switches cannot be overridden) and the matching -Xmx (384m/384m/1536m/2g), placed before -jar so the JVM reads it. Both can be edited in ~/.yano-wallet/<network>/node/node.properties, written as a commented template beside node.log.
  • Scan-index fallback. A node with the scan route but no usable index fell into a history error on every refresh. A 400 "outside complete filter coverage" on a scan from origin — what every chainstate upgraded from a pre-index node returns, because its index starts partway through the chain — now falls back to local history (ADR-043). From a saved cursor the same 400 can mean the index is briefly behind, so it stays an error. A 503 means "no history" only for UTxO state off, coverage.enabled: false, or a reason naming a fresh sync; temporary states (index behind the applied block, storage being replaced, the concurrency limit) stay errors rather than hiding real transactions.
  • Retrying an uncertain submit. After a transport error or 5xx, the retry collided with the reservation its own first attempt left, was recorded as a conflict, and the payment was marked FAILED with its draft discarded — so if the first attempt had landed, a rebuild paid twice. Now the identical signed transaction may be resent (only a different transaction over reserved inputs is a conflict); before a 4xx releases inputs the wallet checks GET /txs/{hash} and reports a transaction already in a block as submitted; a failed check keeps the reservation and asks for a retry; and WalletService.submit treats only a 4xx as a rejection.

Validation

  • ./gradlew :wallet-node-client:test :wallet-core:test :wallet-app:test :wallet-ui:test --console=plain passed. Tests cover ADA/token construction using pending change, pagination, credentials, reservation persistence/cleanup, conflicts and rebuilding, lookup failures, and explicit DevKit isolation.
  • Read-only checks against the running preprod Yano at port 7070 verified an overlay-excluded confirmed input and a pending output with block: null, plus address, credential, address-as-credential, and single-output queries. No user transactions were submitted by these checks.
  • Earlier discovery validation ran the opt-in WalletIndexLiveTest against both JVM and native Yano nodes, covering persisted assets/outpoints, outgoing-only history, fully spent addresses, restart/crash recovery, snapshot reorg recovery, and replacement transactions. Reproduction: docs/wallet-index-live-validation.md. Mainnet-scale history/performance measurements remain deferred.
  • Signer search: DappSignerSearchTest (0/1/2 and change keys, 29/30/49/50 boundaries, known high indexes, account isolation, input vs output/reference ownership, pre-existing witness verification, stake keys, malformed/bounded requests) and Cip30SignerSearchApprovalTest (gate-to-signer wiring, extension accepted/declined, full vs partial sign, session/network/body changes, one-use tickets). Each new commit was also tested on its own: connector-host 24, core 192, app 73, ui 14 tests, all passing. Not yet qualified with a live extension session.
  • Node pin, scan fallback and submit retry: NodeLaunchCommandTest/NodeOptionsTest (argument order, profile and heap from node.properties), WalletScanHistoryTest (400 from origin vs. saved cursor, each 503 reason), PendingInputsTest (resend after an unknown outcome, resend of a transaction already in a block, unverifiable refusal keeps its reservation, self-claimed conflict, a different transaction still refused) and SubmitOutcomeTest (5xx retryable with no FAILED record, 4xx FAILED). Run against the previous code, 6 of the new submit tests fail. Full run: core 194, node-client 121 (5 opt-in live tests skipped), app 73, launcher 44, all passing. Not yet exercised against a live node: the duplicate-returns-200 and out-of-coverage-400 behaviour is taken from the pre16 source.
  • Scan feedback: ScanProgressTrackerTest (unknown tip before ready, resume-cursor interval, completion clearing progress, unreadable records, fraction bounds), WalletScanHistoryTest progress empty after both success and failure, and HistoryMessagesTest for the two empty-state messages and the progress line. Full run: connector-host 24, core 192, node-client 111 (5 opt-in live tests skipped), app 73, ui 17, all passing. The progress line itself has not been seen in the running app — an incremental scan completes in about 9 ms, so showing it needs a scan from origin.

Node dependency and limits

Yano selection requires the newly implemented include_mempool=true listing contract; confirmed-only fallback is not used on lookup failures. Runtime results remain transient, so local reservations and submission conflict handling are retained. There is no custom acknowledgement-header requirement.

The managed node is now pinned to Yano 0.1.0-pre16 (gradle.properties); the pre13 checksums stay in the lock file for an explicit -PyanoVersion override.

Reservations survive wallet restarts but are local to this wallet application. An externally learned conflict with unknown validity cannot currently be released on mempool eviction alone; confirmation clears it. See docs/mempool-coin-selection.md for refresh and failure semantics.

Known open issues

Reported by a code review of this branch and not addressed here; none has been re-verified yet:

  • The pre-signing simulation counts fewer addresses as the wallet's own than the signer can sign for, so a dApp transaction spending a UTxO outside the gap-limited scan can under-report a loss (TxEffectSummariser). On upgraded chainstates an incomplete address scan also makes every simulation "could not be checked".
  • A reservation with TTL 0 (a dApp transaction with no upper validity bound, or a learned conflict owner) is released only when the transaction reaches a block; if it is dropped, its inputs stay excluded from selection indefinitely.
  • pending-inputs.json is not fsynced before its atomic rename; a truncated file after a crash prevents the wallet from connecting until it is deleted.
  • The unlocked DefaultWallet caches accounts in an unsynchronised HashMap but is now used from the signer-search, simulation and dashboard threads.
  • An unused address is accepted only once the first-seen index is at the live tip, so any lag makes the whole address scan — and with it balance, drafts and simulation — fail.
  • CIP-30 getUtxos returns the confirmed view while signing resolves inputs through the selection view, so a dApp can build a transaction the wallet then refuses to sign; chained dApp transactions are refused too.
  • The signer search no longer signs with the primary payment key by default and does not look inside native scripts, so a sponsored mint whose policy needs the wallet's key only via its script is refused.
  • The Dashboard requests history every 5 s with no in-flight guard, queuing behind a long first scan on the single backend executor.
  • Performance: every selection lookup refreshes reservations with network calls under a lock, and the simulation runs a full two-chain address scan inside its 3 s budget.
  • node.properties: maxHeap accepts a bare number (bytes to the JVM) or 0, which stops the node at startup; the file is re-read on each accessor call rather than once per start.

Discovery work relates to bloxbean/yano#119; companion discovery/index node PR: bloxbean/yano#120.

🤖 Generated with Claude Code

@satran004
satran004 marked this pull request as ready for review September 6, 2026 14:26
@satran004 satran004 changed the title feat: use first-seen discovery and persistent wallet scans feat: improve wallet discovery and Yano mempool-aware sends Sep 7, 2026
satran004 and others added 6 commits September 19, 2026 23:22
dApp signTx and signData previously signed only with receive index 0, so a
transaction naming payment keys from other receive or change addresses could
collect only the first witness. ADR-047 defines a bounded search instead:

- Targets come from required_signers and the payment credentials of resolved
  spending/collateral inputs; outputs and reference inputs never imply a key.
- Search the unlocked HD account only: paths already known in this session
  (displayed, funded, or opened via address details), then receive/change
  0-29, and 0-49 only after an explicit user extension. No history or gap
  queries during discovery.
- The approval prompt lists matched derivation paths, the search window and
  unmatched hashes. A one-use ticket binds approval to the exact request
  bytes, session and backend; any change fails closed.
- Sign the original body bytes with only the reviewed keys; pre-existing
  witnesses must verify against that body and are never duplicated.

Adds WalletAddressService.addressDetails (public verification keys and key
hashes per receive address, software and watch-only) as the session call
that makes a high-index path known to the search. Hardware signing is
unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each address on the Receive screen gets a Details action that opens a card
with the payment verification key and key hash, plus a collapsed stake key
section, each with its derivation path and a copy button. Only raw public
keys are exposed, never private material or extended-key chain codes, and
the card notes that sharing them may link activity.

Wired through WalletUiController.addressDetails; the default fails closed
for controllers that do not provide public keys. Viewing an address also
registers its path with the CIP-30 signer search for this session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A first scan walks the whole chain before it can answer — measured at
roughly half a minute against a mainnet node at tip — and the screen
showed nothing at all while it ran, so a working scan and a failure were
indistinguishable. This reports the scan instead of hiding it.

- ScanProgressTracker reads position from the records the scan already
  streams: the ready record states the node's tip, so the interval is
  known before the first block arrives. Kept apart from the scan's own
  bookkeeping so a display value can never affect whether a scan is
  trusted, and records it cannot read are skipped rather than rejected.
- WalletScanHistory publishes the snapshot unsynchronised, because
  transactions() holds the instance lock for the whole scan; a read that
  waited for that lock could only describe a scan already finished. It is
  cleared in a finally, so a failed scan leaves no frozen percentage.
- historyScanStatus() reads it on the common pool, never the backend
  executor the scan itself occupies.
- HistoryScreen shows "Loading history…" immediately, then polls twice a
  second for "Scanning the chain for your transactions — block X of Y".
  A late poll cannot overwrite a page that has arrived.

A node with no scan route and no local record now says "History not
found" rather than "No transactions yet" — the list is empty because
nothing can be read from that node, not because the wallet has sent
nothing. Where local records exist they still show under the existing
"Local history only" chip (ADR-043).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pre16 serves the wallet scan index this branch reads history from. The
checksum lock keeps the pre13 entries, so an explicit
-PyanoVersion=0.1.0-pre13 still verifies.

The managed node is spawned directly rather than through yano.sh, so it
got none of that script's sizing: no resource profile, and a heap left to
JVM ergonomics. It now launches with:

- quarkus.profile=<network>,<sizing>,wallet. The sizing profile (medium by
  default) sets RocksDB caches, open files and the decoded-block queue;
  wallet stays last so its index switches cannot be overridden.
- -Xmx from the profile (384m/384m/1536m/2g, mirroring yano.sh), placed
  before -jar where the JVM reads it. The native binary takes it too.

Both are user-editable in ~/.yano-wallet/<network>/node/node.properties,
written as a commented template beside node.log on first start; bad
values are logged and ignored. -Dyano.wallet.node.sizing-profile and
-Dyano.wallet.node.max-heap do the same for a dev run.
NodeLaunchCommandTest pins the argument order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A node with the scan route but no usable index was reported as a history
error on every refresh, instead of falling back to the wallet's own
record (ADR-043). Classified against the pre16 source:

- 400 "outside complete filter coverage" on a scan from origin: the index
  was enabled after the chainstate synced and starts partway through the
  chain, so it never can. Every chainstate upgraded from a pre-index node
  gets this. From a saved cursor the same 400 can mean the index is
  briefly behind, so that stays an error.
- 503 means "no history" only when the node says so for good: UTxO state
  off, coverage.enabled false, or a reason naming a fresh sync as the
  remedy. The index trailing the applied block, storage being replaced
  and the concurrency limit all pass, so they stay errors; answering them
  with the local list would hide real transactions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
After a transport error or a 5xx, the wallet kept the signed draft for a
retry. The retry then collided with the reservation its own first attempt
had left, and was recorded as a mempool conflict: the payment was marked
FAILED and the draft discarded. If the first attempt had reached the
node, it still landed, and "Rebuild & review" selected other funds,
paying twice.

- PendingInputs: only a different transaction over reserved inputs is a
  conflict. The identical signed transaction may be resent; the ledger
  applies it at most once, and Yano answers one already in its mempool
  with 200.
- YanoTransactionProcessor: before a 4xx releases inputs, it asks
  GET /txs/{hash}. A resend of a transaction already in a block is
  refused for spending its own inputs, and is reported as submitted. If
  the lookup fails the outcome stays unknown: the reservation is kept and
  the caller is told to retry. A conflict naming this same transaction is
  treated as already in the mempool.
- WalletService.submit: only a 4xx is a rejection. A 5xx is an unknown
  outcome and now keeps the draft (RetryableSubmitException), like a
  dropped connection, instead of recording FAILED.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@satran004
satran004 merged commit 3b5e32f into main Oct 5, 2026
1 check passed
@satran004
satran004 deleted the feat/119-wallet-indexes branch October 5, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant