Skip to content

refactor(core): read headers, state updates and classes through typed buckets - #4235

Open
danielntmd wants to merge 1 commit into
mainfrom
danielntmd/cbor-typed-buckets
Open

danielntmd wants to merge 1 commit into
mainfrom
danielntmd/cbor-typed-buckets

Conversation

@danielntmd

@danielntmd danielntmd commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Route the header, state update and class Get, Write and Delete accessors through their typed buckets.

  • ClassBucket now uses the Cbor serializer, which matches how the class is stored.
  • Bucket.GetPtr decodes into a heap value, so accessors that return a pointer do not copy it a second time.

Testing

  • Matched against old decode through using new accessors, covered every 20th header, state update, and class up to block 12,945,535.

… buckets

Route the header, state update and class Get, Write and Delete accessors
through their typed buckets. ClassBucket now uses the Cbor serializer,
which matches how the class is stored.

Bucket.GetPtr decodes into a heap value, so accessors that return a pointer
do not copy it a second time.
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @danielntmd's task in 17m 40s —— View job


Review complete

  • Gather context (diff, changed files)
  • Review db/typed/bucket.go changes (GetPtr)
  • Review core/accessors.go changes
  • Review core/state/accessors.go changes
  • Review core/typed_buckets.go changes (Cbor serializer for ClassBucket)
  • Review db/typed/bucket_test.go changes
  • Post findings

Summary: Clean, mechanical refactor. No blockers or important issues found.

Notes from the review (no action needed):

  • Bucket.GetPtr (db/typed/bucket.go:43) correctly allocates new(V) on the heap and returns nil, err on failure, matching the pre-existing var x *T; ...; return x, err pattern it replaces — including returning nil rather than a half-populated struct when the key is missing.
  • Switching ClassBucket (core/typed_buckets.go:25-29) from value.Binary[...] to value.Cbor[...] is correct and not a storage-format change: the old core.GetClass/WriteClass already called the generic cbor.Marshal/cbor.Unmarshal on *DeclaredClassDefinition directly (not .MarshalBinary()/.UnmarshalBinary()), so the on-disk bytes are unchanged — value.Cbor[V]() just makes the typed bucket match what was already being written. value.Binary[V]() would have bypassed the CBOR byte-string wrapping fxamacker's lib applies for encoding.BinaryMarshaler types and produced different bytes, so the bucket was correctly left unintegrated (TODO) until this change.
  • core/state/accessors.go's GetClass/WriteClass/DeleteClass now delegate to core.ClassBucket instead of duplicating the cbor encode/decode logic — removes duplication without behavior change.
  • Key casts like (*felt.ClassHash)(classHash) are safe since felt.ClassHash and felt.Felt are both Typed[K] fp.Element aliases with identical underlying type, and this conversion pattern already exists elsewhere in the codebase (e.g. rpc/v10/transaction.go:169).
  • Test additions in db/typed/bucket_test.go adequately cover GetPtr's success/not-found paths alongside the existing Get assertions.

@github-actions

Copy link
Copy Markdown
Contributor

Title

refactor(core): read headers, state updates and classes through typed buckets


PR Type

Enhancement, Tests


Description

  • Route class, state update, and header accessors through typed buckets

  • Switch ClassBucket to CBOR serializer matching stored format

  • Add Bucket.GetPtr decoding into a heap value to avoid extra copies

  • Extend typed bucket tests to cover GetPtr behavior


File Walkthrough

Relevant files
Enhancement
accessors.go
Accessors read and write via typed buckets                             

core/accessors.go

  • Replace manual CBOR marshal/unmarshal in class, state update, and
    block header accessors with ClassBucket,
    StateUpdatesByBlockNumberBucket, and BlockHeadersByNumberBucket calls
  • Use new GetPtr for read accessors returning pointers (GetClass,
    GetStateUpdateByBlockNum, GetBlockHeaderByNumber)
  • Delegate write/delete accessors to bucket Put and Delete
+9/-37   
accessors.go
State class accessors delegate to ClassBucket                       

core/state/accessors.go

  • WriteClass, DeleteClass, and GetClass now delegate to core.ClassBucket
    methods instead of manual CBOR encoding and raw key handling
  • Remove the now-unused utils/cbor import
+3/-18   
typed_buckets.go
ClassBucket uses CBOR; integration TODOs removed                 

core/typed_buckets.go

  • Change ClassBucket value serializer from value.Binary to
    value.Cbor[DeclaredClassDefinition], matching on-disk class encoding
  • Remove TODO: Integrate this bucket comments from ClassBucket,
    BlockHeadersByNumberBucket, and StateUpdatesByBlockNumberBucket now
    that they are wired in
+1/-4     
bucket.go
Add GetPtr to typed Bucket                                                             

db/typed/bucket.go

  • Add GetPtr method that unmarshals into a heap-allocated value and
    returns *V, avoiding a second copy for pointer-returning accessors
  • Return nil pointer on read errors
+11/-0   
Tests
bucket_test.go
Cover GetPtr in bucket tests                                                         

db/typed/bucket_test.go

  • Extend assertExists to verify GetPtr returns the expected value
  • Extend assertNotExists to verify GetPtr returns db.ErrKeyNotFound and
    a nil pointer
+8/-0     

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.44444% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.12%. Comparing base (00a3977) to head (3ced76a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
core/accessors.go 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4235      +/-   ##
==========================================
- Coverage   81.25%   81.12%   -0.14%     
==========================================
  Files         479      479              
  Lines       35591    35568      -23     
==========================================
- Hits        28921    28855      -66     
- Misses       6664     6707      +43     
  Partials        6        6              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@danielntmd
danielntmd deployed to Development October 30, 2026 00:20 — with GitHub Actions Active
@danielntmd
danielntmd requested a review from infrmtcs October 30, 2026 12:43
Comment thread db/typed/bucket.go
return value, err
}

func (b Bucket[K, V, KS, VS]) GetPtr(database db.KeyValueReader, key K) (*V, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should defer the reference to the caller, if we want to use return by value by default.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to keep only the return by value, we can drop this and accept the cost of an extra allocation for callers that return pointers.

Comment thread core/typed_buckets.go
db.Class,
key.ClassHash,
value.Binary[DeclaredClassDefinition](),
value.Cbor[DeclaredClassDefinition](),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although the existing typed bucket code was incorrect and the new code is, I think we should replace this by something like BinaryCbor.
One of the reason is because this is an implicit fxamacker feature to use MarshalBinary and wrap it as a CBOR bytestring if the type implements MarshalBinary. This means if we change the cbor library to anything that doesn't support this behaviour, it will be incorrect.
Other affected types are AggregatedBloomFiltersBucket and RunningEventFilterBucket which have the same behaviour.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can call MarshalBinary ourselves and let fxamacker write and read the byte slice, but there is a class read cost (2x bytes allocated on class reads) when unmarshalling into a byte slice.

Alternatively, we can either pin this behavior through the golden tests for the AggregatedBloomFiltersBucket and RunningEventFilterBucket or we can create a zero copy unmarshal call specifically for that header but that would add another API to cbor (and ~30 lines to own CBOR header parsing, which I would want to avoid).

This branch was successfully deployed

1 active deployment
Development — 3ced76ab Deployed Oct 30, 2026 by danielntmd via dev_deploy_test / deploy #6227
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants