Skip to content

fix(freshness): error on NULL loaded_at instead of reporting a ~56-year staleness - #16587

Open
kiwamizamurai wants to merge 2 commits into
dbt-labs:mainfrom
kiwamizamurai:fix/freshness-null-max-loaded-at
Open

kiwamizamurai wants to merge 2 commits into
dbt-labs:mainfrom
kiwamizamurai:fix/freshness-null-max-loaded-at

Conversation

@kiwamizamurai

@kiwamizamurai kiwamizamurai commented Oct 3, 2026 •

Copy link
Copy Markdown

Resolves #16489

Problem

When MAX(loaded_at_field) (or a loaded_at_query result) is SQL NULL — e.g. the source table has no rows — calculate_freshness_common read the value with PrimitiveArray::value(0). That accessor ignores the null bitmap and returns whatever is in the buffer slot, which is 0 (the Unix epoch). dbt source freshness therefore reported a specific-looking but meaningless staleness:

Stale source my_source.empty_table (last updated 56years 8months 26days 8h 2s ago)

Solution

Add a small helper, extract_non_null_nanos, that casts the column to nanoseconds (as before) and checks is_null(0) before calling .value(0). If the value is NULL it returns an explicit error instead of computing an age from garbage. It is used for both max_loaded_at and snapshotted_at (the latter defensively).

No public types, JSON artifacts, or CLI flags change.

Behavior of the error at the call sites (unchanged code, relying on the existing per-node handling in run_freshness):

  • dbt source freshness: the node is reported as an error with the new message.
  • dbt build / dbt run (freshness collected as part of state-aware orchestration): a warning is emitted and the source is treated as updated, so dependents rebuild. Previously an empty table was read as the epoch.

Alternative considered: a dedicated "no data" FreshnessStatus. I did not do it here because it would touch run_results / telemetry / artifacts; happy to follow up if maintainers prefer that direction.

Testing

Added unit tests for the helper (NULL → error; present value → correct nanoseconds, via a Timestamp(Microsecond, UTC) column as in the issue's reproduction).

Checklist

  • I have read the contributing guide and understand what's expected of me.
  • I have run this code in development, and it appears to resolve the stated issue.
  • This PR includes tests, or tests are not required or relevant for this PR.
  • This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.

…Unix epoch

When `MAX(loaded_at_field)` is NULL (e.g. the source table has no rows),
`calculate_freshness_common` read the value with `PrimitiveArray::value(0)`,
which ignores the null bitmap and returns 0 (1970-01-01). `dbt source freshness`
then reported a bogus ~56 year staleness.

Check `is_null(0)` before reading the value and return an explicit error for
both `max_loaded_at` and `snapshotted_at`.

Resolves dbt-labs#16489

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@cla-bot cla-bot Bot added the cla:yes label Oct 3, 2026
codescene-delta-analysis[bot]

This comment was marked as outdated.

…elper

Keeps `calculate_freshness_common` shorter than before (CodeScene Large Method).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kiwamizamurai
kiwamizamurai marked this pull request as ready for review October 3, 2026 22:31
@kiwamizamurai
kiwamizamurai requested a review from a team as a code owner October 3, 2026 22:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v2 Bug] Source freshness reports a bogus ~56-year staleness (silently falls back to Unix epoch) when MAX(loaded_at_field) is NULL on an empty table

1 participant