Skip to content

feat(logging): add transaction commit lifecycle logging - #890

Merged
wgtmac merged 4 commits into
apache:mainfrom
kamcheungting-db:logging-commit-retry
Sep 19, 2026
Merged

wgtmac merged 4 commits into
apache:mainfrom
kamcheungting-db:logging-commit-retry

Conversation

@kamcheungting-db

@kamcheungting-db kamcheungting-db commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Add operational logging to Transaction::Commit so retry behavior and commit
outcomes are easier to diagnose.

Changes

  • Log a WARN for each retry, including the table identifier, attempt number,
    and previous error.
  • Log an INFO only after a real metadata commit succeeds.
  • Include the number of attempts and newly committed snapshot details in the
    success log when available.
  • Do not report empty or no-op transactions as successful commits.
  • Do not classify CommitStateUnknown as a confirmed failure.
  • Format snapshot details through the lazy logging path so logging failures
    cannot affect the commit result.

The retry policy and retry decisions are unchanged.

Validation

  • Built table_update_test
  • Ran the transaction retry and no-op transaction tests
  • 9 tests passed
  • clang-format, pre-commit, and git diff --check passed

@kamcheungting-db kamcheungting-db changed the title feat(logging): log transaction commit retries and final outcome feat(logging): log transaction commit lifecycle (retries, success, snapshot adds) Aug 17, 2026
Comment thread src/iceberg/transaction.cc Outdated
Comment thread src/iceberg/transaction.cc Outdated
Comment thread src/iceberg/table_metadata.cc Outdated
@kamcheungting-db

Copy link
Copy Markdown
Contributor Author

@wgtmac could you help me review this pr one more time?

Transaction::Commit runs through a retry runner but was completely silent, so
operators could not tell whether a commit was retrying on a transient conflict or
had failed permanently. This is the first real adoption of the logging component
in the commit path.

- WARN on each genuine retry (the runner only re-invokes the task when it decides
  to retry, so attempt > 1 marks a real retry), carrying the prior error.
- INFO when a commit finally succeeds after > 1 attempt.
- ERROR when retries are exhausted, with the attempt count and final error.

Tests (TransactionRetryTest, via a CapturingLogger installed with
ScopedDefaultLogger): assert the retry WARN + success INFO on a retry-then-succeed
commit, and the exhaustion ERROR on an always-conflicting commit.

Co-authored-by: Isaac
Extend the commit-path logging beyond retries:
- Transaction::Commit now logs an INFO on every successful commit (previously only
  after a retry). When the commit advanced the current snapshot (a data commit) the
  message names the snapshot id and operation; metadata-only commits report a plain
  success.
- TableMetadataBuilder::AddSnapshot logs a DEBUG naming the snapshot id and sequence
  number when a snapshot is added to the metadata.

Tests: single-attempt commit emits the success INFO with no retry WARN
(TransactionRetryTest.CommitSuccessEmitsInfoLog); AddSnapshot emits the DEBUG
(TableMetadataBuilderTest.AddSnapshotEmitsDebugLog).

Co-authored-by: Isaac
@wgtmac
wgtmac force-pushed the logging-commit-retry branch from f757a00 to b2be450 Compare September 19, 2026 10:39

@wgtmac wgtmac left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refined the commit logging to keep it aligned with the actual commit behavior. Empty or no-op transactions no longer emit a misleading success message, retry logs now use RetryRunner's attempt counter and include the table identifier, and snapshot details are formatted lazily through the logging path so logging failures cannot affect the commit result. This keeps the operational logs trustworthy during retries without changing the commit semantics.

@wgtmac wgtmac changed the title feat(logging): log transaction commit lifecycle (retries, success, snapshot adds) feat(logging): add transaction commit lifecycle logging Sep 19, 2026
@wgtmac
wgtmac merged commit a1c947b into apache:main Sep 19, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants