Skip to content

perf(misk-aws2-dynamodb): truncate tables on external fixture reset instead of drop/recreate - #3864

Open
rnc505 wants to merge 2 commits into
cashapp:masterfrom
rnc505:rnc505/external-dynamodb-truncate-on-reset
Open

perf(misk-aws2-dynamodb): truncate tables on external fixture reset instead of drop/recreate#3864
rnc505 wants to merge 2 commits into
cashapp:masterfrom
rnc505:rnc505/external-dynamodb-truncate-on-reset

Conversation

@rnc505

@rnc505 rnc505 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Background

  • ExternalTestDynamoDbClientModule currently drops and recreates every configured table before each test.
  • Against a shared DynamoDB Local server, repeated table DDL can add substantial fixed latency to every test.
  • In early testing against a service suite with thousands of tests, truncating rows instead reduced the test phase from approximately 17 minutes to approximately 7 minutes.

Overview of Changes

  • Recreate configured tables once per injector to remove state left by earlier processes.
  • Truncate rows on later resets using projected primary keys and batched deletes.
  • Handle composite and reserved key names, scan pagination, and bounded retries for unprocessed writes.
  • Preserve the module's existing public API and binding layout.
  • Add ResetStrategy.DROP_RECREATE as a one-line opt-out for consumers using DynamoDB Streams or mid-test table DDL, whose state cannot be isolated by truncating rows.
  • Add coverage for initial state, more than 25 rows, unchanged table identity, GSI cleanup, and drop/recreate resets.

Testing Performed

  • bin/gradle :misk-aws2-dynamodb:test --warn
  • bin/gradle :misk-aws2-dynamodb:check --warn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant