feat: open LargeSmtForest backend API for external implementations - #3413
Conversation
|
Automated check (CONTRIBUTING.md) Findings:
Recommendations:
Next steps:
|
|
Ideally we would like this in crypto 0.28.1 so we can get to use this soon. Since crypto has been moved to this repository I'm not entirely sure how crypto releases are managed. I pointed this PR to |
huitseeker
left a comment
There was a problem hiding this comment.
This works! This will get published as 0.28.1 and does not need a major bump.
|
@JereSalo This will normalize as soon as we release the VM at 0.28, but what you did is just fine. |
|
Thanks for the review. I had to add one more little change that I noticed was necessary while implementing the |
miden-crypto 0.28.1 is still unpublished, so the patch remains, but it now pins the official 0xMiden/miden-vm repo at the squash-merge of 0xMiden/miden-vm#3413 instead of a mutable personal-fork branch.
Description
The
Backendtrait forLargeSmtForestis public but cannot be implemented outside this crate, becausecompute_mutationsandapply_mutationsmust returnLineageMutationandAppliedLineageMutationvalues whose fields are private and whose constructors arepub(crate). This PR makes both constructors public and adds aLineageId::as_bytesgetter so backends can persist lineage identifiers.It also adds a
MutationSet::from_partsconstructor. An external backend has to produce the forward and reverse mutation sets carried byAppliedLineageMutation, and sinceMutationSetfields are private the only way to build them today is to load the full tree into memory and runSmt::compute_mutations/apply_mutations_with_reversion, which makes update cost scale with tree size instead of change size. With the constructor, a backend can compute mutations against its persisted tree data (affected leaves and stored subtree nodes only) and assemble both sets directly, the same way the built-in RocksDB backend does internally. In the SQLite backend prototype this took a single-entry committed update on a 1,000,000-entry tree from ~30 s to ~1 ms.We need this change because in 0xMiden/rust-sdk#2315 we want to use the LargeSmtForest but with an SQLite backend. And it's very likely that we'll want to do the same in the web SDK with IndexedDB.
In order to make use of this change we should also bump miden-crypto, so it'd be good to have this in 0.28.1 since miden-protocol currently pins
^0.28.