Skip to content

Replace O(n²) lookups with Map-based atoms in schema state layer - #1744

Merged
kmcginnes merged 1 commit into
aws:mainfrom
kmcginnes:map-based-preference-atoms
Apr 30, 2026
Merged

kmcginnes merged 1 commit into
aws:mainfrom
kmcginnes:map-based-preference-atoms

Conversation

@kmcginnes

@kmcginnes kmcginnes commented Apr 30, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Replace O(n²) .find() loops with O(1) Map-based lookups across the schema state layer. At 70k vertex types, mergeConfiguration goes from 18.8 seconds to 162 milliseconds (116x faster).

  • Add vertexPreferencesAtom and edgePreferencesAtom with .get(type) accessor that returns resolved preferences via O(1) Map lookup with default fallback
  • Simplify vertexPreferenceByTypeAtom / edgePreferenceByTypeAtom to delegate to the new Map atoms
  • Simplify useAllVertexPreferences / useAllEdgePreferences by removing intermediate useStoredGraphPreferences wrapper
  • Pre-index vertex, edge, and preference arrays into Maps in mergeConfiguration, converting O(n²) .find() calls to O(1) .get() lookups
  • Convert mergeAttributes to use Map-based lookups
  • Replace lodash.cloneDeep with destructuring spread in patchToRemoveDisplayLabel, avoiding 70k deep clones
  • Replace lodash.uniq() with native Set for deduplication
  • Fix circular import in storageAtoms.ts

Benchmarks (mergeConfiguration, 3 attributes per type)

Types Old (mean) New (mean) Speedup
20 0.048ms 0.029ms 1.7x
1,000 5.98ms 1.72ms 3.5x
10,000 506ms 20.9ms 24x
70,000 18,770ms 162ms 116x

Validation

  • All existing tests pass with no changes to assertions — behavior is preserved
  • New tests verify patchToRemoveDisplayLabel does not mutate its input
  • New tests verify the Map-based preference atoms return stored preferences for known types and defaults for unknown types
  • Benchmarked with a synthetic 70k-type schema

Related Issues

Check List

  • I confirm that my contribution is made under the terms of the Apache 2.0 license.
  • I have verified pnpm checks passes with no errors.
  • I have verified pnpm test passes with no failures.
  • I have covered new added functionality with unit tests if necessary.
  • I have updated documentation if necessary.

- Add vertexPreferencesAtom and edgePreferencesAtom with .get(type)
  accessor that returns resolved preferences via O(1) Map lookup
- Simplify vertexPreferenceByTypeAtom/edgePreferenceByTypeAtom to
  delegate to the new Map atoms
- Simplify useAllVertexPreferences/useAllEdgePreferences by removing
  intermediate useStoredGraphPreferences wrapper
- Pre-index vertex, edge, and preference arrays into Maps in
  mergeConfiguration, converting O(n²) .find() calls to O(1) lookups
- Convert mergeAttributes to use Map-based lookups
- Replace lodash cloneDeep with destructuring spread in
  patchToRemoveDisplayLabel, avoiding 70k deep clones
- Replace lodash uniq() with native Set for deduplication
- Fix circular import in storageAtoms.ts
- Add tests for new atoms and patchToRemoveDisplayLabel immutability

Part of aws#1678
@kmcginnes
kmcginnes marked this pull request as ready for review April 30, 2026 22:43
@kmcginnes
kmcginnes merged commit 7f4e840 into aws:main Apr 30, 2026
3 checks passed
@kmcginnes
kmcginnes deleted the map-based-preference-atoms branch April 30, 2026 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eliminate O(n²) operations in the schema state layer

2 participants