feat(chain)!: taint-aware CanonicalView::balance - #2246
Conversation
Adds classify_outpoints, which decides per-output whether it's settled, immature, or pending (trusted/untrusted) based on its unsettled ancestry. Taint is resolved with a memoized ancestry walk, so ancestors shared by several outputs are only walked once. Co-authored-by: 志宇 <hello@evanlinjin.me> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
balance becomes a fold over classify_outpoints. A pending output's trust is now taken from its unsettled ancestry (untrusted if any unsettled ancestor taints), not from a per-output flag. BREAKING: balance takes does_taint and is_settled instead of trust_predicate and min_confirmations, and plain OutPoints instead of (identifier, outpoint) pairs. Co-authored-by: Dmenec <dmenec@proton.me> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- taint propagates through a pending output's unsettled ancestry - is_settled alone decides the settled boundary, even for unconfirmed outputs - taint never crosses a settled ancestor - an immature coinbase is classified apart from a settled output - two UTXOs sharing a tainting ancestor are both untrusted (shared taint cache) - classify_outpoints skips spent and out-of-view outpoints Co-authored-by: 志宇 <hello@evanlinjin.me> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Times the memoized classification over fan-in, disjoint, untrusted, diamond and deep-taint-mid-chain graphs at a range of widths and depths.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2246 +/- ##
==========================================
+ Coverage 78.32% 78.36% +0.03%
==========================================
Files 30 30
Lines 5934 5995 +61
Branches 281 282 +1
==========================================
+ Hits 4648 4698 +50
- Misses 1210 1221 +11
Partials 76 76
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thinking about some cases where settledness makes a transaction fall into a different I think it might be clearer to call them |
|
@Dmenec Good point. What do you think about this? pub enum Eligibility {
Settled,
Immature,
Unsettled(Trust),
}
pub enum Trust {
Trusted,
Untrusted,
}I think this increases clarity and makes it a bit easier for call sites that only care about whether it's unsettled or not. |
evanlinjin
left a comment
There was a problem hiding this comment.
Thanks for pushing this forward - this is looking great.
I haven't looked too hard into the tests yet, follow-up reviews will come.
| /// | ||
| /// This is a *chain-level* classification. It deliberately knows nothing about wallet policies such | ||
| /// as locked or reserved coins; callers layer their own categories on top. |
There was a problem hiding this comment.
| /// | |
| /// This is a *chain-level* classification. It deliberately knows nothing about wallet policies such | |
| /// as locked or reserved coins; callers layer their own categories on top. |
Let's remove this - I don't think it's useful.
| /// as locked or reserved coins; callers layer their own categories on top. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] | ||
| pub enum Eligibility { | ||
| /// A confirmed output unlikely to be replated. |
There was a problem hiding this comment.
| /// A confirmed output unlikely to be replated. | |
| /// A confirmed output unlikely to be replaced. |
| .into_iter() | ||
| .filter_map(|op| self.txout(op)) | ||
| .filter(|txo| txo.spent_by.is_none()) | ||
| .collect::<Vec<_>>(); |
| // it once instead of redoing the same work for each outpoint. | ||
| let mut cache = HashMap::<Txid, bool>::new(); | ||
| let results: Vec<_> = utxos | ||
| .into_iter() |
There was a problem hiding this comment.
Can remove this once utxos is an iterator.
| .into_iter() |
| if txout.is_mature(tip) { | ||
| Eligibility::Settled | ||
| } else { | ||
| Eligibility::Immature | ||
| } |
There was a problem hiding this comment.
The coinbase maturity check lives inside the is_settled check - this is wrong. Unsettled is not the same as unconfirmed.
Example: A confirmed transaction can be unsettled because the caller requires 3 confirmations to be classified as settled. With the current logic, this transaction will become "pending" instead of "immature".
There was a problem hiding this comment.
So any unconfirmed coinbase tx with an owned script pubkey should be treated as Immature, not TrustedPending, even if it doesn't have a single confirmation?
There was a problem hiding this comment.
Hopefully there shouldn't be such a thing as an "unconfirmed coinbase" - the canonicalization algorithm should have considered those as non-canonical! If not, let's file a bug.
There was a problem hiding this comment.
Not even if you assume it canonical?
There was a problem hiding this comment.
Tried it with a coinbase without an anchor (which, as @evanlinjin said, shouldn't be possible on a real chain, but you can still construct it in a test), and if you assume it canonical it does end up in the set as unconfirmed.
bdk/crates/chain/src/canonical.rs
Lines 132 to 137 in 8897579
As you can see above, it falls back to false there, so such an output is treated as immature anyway.
I agree with moving the maturity check out of is_settled, so it stays Immature regardless of settledness.
| } | ||
| } | ||
| Frame::Exit(txid) => { | ||
| let c_tx = self.tx(txid).expect("present, checked in Enter"); |
There was a problem hiding this comment.
If we change the signature of Exit to also contain the transaction itself, we can avoid calling self.tx() again.
| // Not canonical | ||
| cache.insert(txid, false); |
There was a problem hiding this comment.
If a transaction does not exist in the CanonicalView, it just means the wallet has sparse data. Therefore, "not canonical" is incorrect.
We can guarantee that all transactions are canonical at this point as in line 440, we filtered by self.txout().
Proposal
Assuming we introduce the Eligibility::Unsettled(Trust) variant, we can add another variant to the Trust enum: Trust::Unknown.
The cache type will become HashMap<Txid, Trust> where Untrusted beats Unknown and Unknown beats Trusted.
There was a problem hiding this comment.
We can guarantee that all transactions are canonical at this point as in line 440, we filtered by
self.txout().
You mean we consider txs for which its txid appear as parents of a tx in the CanonicalView as canonical?
Proposal
Assuming we introduce the
Eligibility::Unsettled(Trust)variant, we can add another variant to theTrustenum:Trust::Unknown.The
cachetype will becomeHashMap<Txid, Trust>whereUntrustedbeatsUnknownandUnknownbeatsTrusted.
Do you expect wallets to also handle this category as "unresolved" balance? Or it's just an intermediate category to trigger request for those txids and be able to resolve the whole trustness of the txs chains?
In my mind we will end up considering this as Untrusted because we cannot claim that we know if these are tainted or not.
There was a problem hiding this comment.
You mean we consider txs for which its txid appear as parents of a tx in the CanonicalView as canonical?
Yes. However note that CanonicalView is a sparse view as TxGraph itself is also sparse. Therefore, the comment and logic here is wrong since we are visiting ancestors of canonical transactions, so anything we visit must also be canonical.
Do you expect wallets to also handle this category as "unresolved" balance? Or it's just an intermediate category to trigger request for those txids and be able to resolve the whole trustness of the txs chains?
I think it's good to be transparent to the caller. Some callers may want to find those ancestors. Other callers may prefer to just treat unknown-trust transactions as untrusted.
In my mind we will end up considering this as Untrusted because we cannot claim that we know if these are tainted or not.
I think you're probably correct. I also assume that for the majority of cases, the caller would just want to treat those transactions as untrusted. However, a transparent API is a more understandable API.
| .collect(); | ||
| results.into_iter() |
There was a problem hiding this comment.
There is a way to write this to not .collect and then call .into_iter on it again.
| if txout.is_mature(tip) { | ||
| Eligibility::Settled | ||
| } else { | ||
| Eligibility::Immature | ||
| } |
There was a problem hiding this comment.
So any unconfirmed coinbase tx with an owned script pubkey should be treated as Immature, not TrustedPending, even if it doesn't have a single confirmation?
| // Not canonical | ||
| cache.insert(txid, false); |
There was a problem hiding this comment.
We can guarantee that all transactions are canonical at this point as in line 440, we filtered by
self.txout().
You mean we consider txs for which its txid appear as parents of a tx in the CanonicalView as canonical?
Proposal
Assuming we introduce the
Eligibility::Unsettled(Trust)variant, we can add another variant to theTrustenum:Trust::Unknown.The
cachetype will becomeHashMap<Txid, Trust>whereUntrustedbeatsUnknownandUnknownbeatsTrusted.
Do you expect wallets to also handle this category as "unresolved" balance? Or it's just an intermediate category to trigger request for those txids and be able to resolve the whole trustness of the txs chains?
In my mind we will end up considering this as Untrusted because we cannot claim that we know if these are tainted or not.
| 0, | ||
| env.indexer.outpoints().iter().map(|(_, op)| *op), | ||
| |_| false, | ||
| |pos| pos.is_confirmed(), |
There was a problem hiding this comment.
The new closure isn't equivalent to the old one. You must check the indexing of the script pubkey.
| graph.index.outpoints().iter().cloned(), | ||
| |_, txout| trusted_spks.contains(&txout.txout.script_pubkey), | ||
| 0, | ||
| graph.index.outpoints().iter().map(|(_, op)| *op), |
There was a problem hiding this comment.
I think the below tests assertions shouldn't be changed to match the new behavior, but the test be reworked to match the assertions. If the test was expecting some untrusted_pending balance, it should be changed to still produce that untrusted pending balance.
| }, | ||
| |pos| pos.is_confirmed(), | ||
| ) | ||
| .map(|(txout, eligibility)| (txout.outpoint, eligibility)) |
There was a problem hiding this comment.
Does it make sense to have impl From<Iter<TxOut, Elegibility>>> for Balance? Is it possible?
| let mut tx_graph = TxGraph::<ConfirmationBlockTime>::default(); | ||
| let spk = ScriptBuf::new(); | ||
|
|
||
| // Coinbase confirmed at height 1; far below `COINBASE_MATURITY` → immature. |
There was a problem hiding this comment.
Please, remove any non-ASCII characters from output: → (use ->)
|
|
||
| /// `classify_outpoints` distinguishes an immature coinbase from a settled output. | ||
| #[test] | ||
| fn test_classify_immature_and_settled() { |
There was a problem hiding this comment.
After evan's review, we should have a test that contains a mature input at current height, and is_settled to aim for 3 confs, and check mature is still mature.
|
|
||
| /// `classify_outpoints` skips outpoints that are already spent or absent from the view. | ||
| #[test] | ||
| fn test_classify_skips_spent_and_unknown() { |
There was a problem hiding this comment.
We should consider the other kind of unknown evan mentioned, parent txids, that aren't indexed, and are the parent root of an unconfirmed tx chain.
Description
Takes over #2235 (thanks @evanlinjin for the go-ahead). It reworks
CanonicalView::balanceto derive trust from an output's unconfirmed ancestry, and addsclassify_outpoints, a per-output spend-eligibility classifier thatbalancebecomes a thin fold over.The old
balancedecided trust using a per-outputtrust_predicate, which cannot express transitive trust. As a result, owned outputs whose unconfirmed ancestry included foreign coins were counted as trusted. That is the root cause of the wallet trust-classification bugs (bitcoindevkit/bdk_wallet#16, bitcoindevkit/bdk_wallet#273).The API now takes two separate predicates, one per concern:
does_taint(&tx)- should this transaction be considered tainted? (e.g., because it spends aforeign unconfirmed output)
is_settled(&pos)- do we consider this chain position settled / final? (generalizesmin_confirmations)For each unspent output,
classify_outpointsreports its chain-level spend eligibility:Settledif considered settled according tois_settledImmaturefor a coinbase output that has not yet maturedTrustedPendingif the output is pending but trustedUntrustedPendingif the output itself, or any of its unconfirmed ancestors, is taintedbalancethen sums each output's value into the bucket corresponding to itsEligibility.Notes to the reviewers
Balance::confirmedand did not rename it tosettled. That rename is out of scope here. The folding logic is slated to move intobdk_wallet, and the currentbalancefunction inchainwill eventually be deprecated.balancealso drops theOgeneric and now takes plainOutPoints, since the taint predicate operates on transactions rather than per-outpoint associated data.does_taintis evaluated at most once per transaction.classify_outpoints. Left for a follow-up.Changelog notice
CanonicalView::balancenow takesdoes_taint: impl FnMut(&CanonicalTx) -> boolandis_settled: impl Fn(&ChainPosition) -> boolinstead of a per-output trust predicate andmin_confirmations, and plainOutPoints instead of(identifier, outpoint)pairs (this dropsthe
Ogeneric). Trust is now derived from an output's unconfirmed ancestry.CanonicalView::classify_outpointsand theEligibilityenum.Checklists
All Submissions:
New Features:
Bugfixes: