TRT-2856: Extend partition start bound to cover outlier job start times

[petr-muller]

Summary

• sippy-backup CronJob pods (fresh-DB sippy load --init-database --release=4.20) have been failing repeatedly with:

  inserting prow_job_run_tests: ERROR: no partition of relation "prow_job_run_tests_p4_20" found for row (SQLSTATE 23514)
  

• Root cause: ensurePartitions() only creates daily partitions covering [loadSince-1day, now+2days]. Jobs that Prow eventually marks aborted after getting stuck can report a StartTime days before their completion time (the field the BigQuery fetch query actually filters on), so such a job can be fetched while its StartTime falls outside the partition window. Confirmed one such job directly: periodic-ci-openshift-release-main-nightly-4.20-e2e-agent-single-node-ipv6-conformance (build 2076094714775343104), started 2026-07-12, not marked complete (aborted) until 2026-07-15 — an 80 hour gap, well past the existing 1-day grace.
• This isn't a one-off: over the last 30 days, several aborted jobs (all releases) had start→completion gaps beyond 24h, with the worst observed around 83h.
• Extend the partition start bound to the earliest Status.StartTime actually present in the fetched job set (with the existing loadSince - 1 day as a floor), so partitions always cover the real data about to be written, regardless of how anomalous a single job's timestamp is.

This is a follow-up to #3798 (which stops one bad batch from aborting/cancelling the entire load) — this PR addresses the actual partition-range gap that causes the batch to fail in the first place.

Test plan

• go build ./pkg/dataloader/prowloader/...
• go vet ./pkg/dataloader/prowloader/...
• go test ./pkg/dataloader/prowloader/...
• golangci-lint run ./pkg/dataloader/prowloader/... (no new findings)
• Added TestPartitionStartDate covering: no jobs, jobs within the grace window, zero-value start times, and an outlier job extending the bound

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved historical job coverage by extending the partition start bound when workflow runs began earlier than the configured load window.
  • Partition selection now considers actual job start times (ignoring zero/unknown values) and falls back to the existing grace-period behavior when none are available.
• Tests
  • Added unit coverage for the partition-start calculation, including grace-window, missing data, and earlier-than-window scenarios.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Walkthrough

The Prow loader now extends partition creation to earlier non-zero job start times while retaining a one-day grace period. Load passes fetched jobs into partition setup, with table-driven tests covering the date-boundary cases.

Changes

Prow partition bounds

Layer / File(s)	Summary
Compute and wire partition start date pkg/dataloader/prowloader/prow.go, pkg/dataloader/prowloader/prow_test.go	partitionStartDate selects the earliest applicable date, Load supplies fetched jobs to ensurePartitions, and tests cover fallback, zero timestamps, and earlier job starts.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: sosiouxme, xueqzhan

🚥 Pre-merge checks | ✅ 20 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Structure And Quality	⚠️ Warning	TestPartitionStartDate is single-purpose, but its asserts use plain assert.True without diagnostic failure text, so failures won't clearly identify the case.	Switch to assert.Equal/NoError with explicit failure text per subtest (or add messages to assert.True) so failures show the scenario and expected vs actual values.

✅ Passed checks (20 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Go Error Handling	✅ Passed	New partition code wraps errors with %w, adds no panic, and only existing _=row.Scan is explicitly justified by comment.
Sql Injection Prevention	✅ Passed	PASS: The patch only changes partition date calculation/call timing; no SQL strings, concatenation, or parameter handling were added or modified.
Excessive Css In React Should Use Styles	✅ Passed	PR only touches Go loader logic and tests; no React/JSX/CSS files or inline style objects are involved.
Test Coverage For New Features	✅ Passed	Added TestPartitionStartDate covers the new pure helper and regression cases (nil, grace window, zero StartTime, outlier start).
Single Responsibility And Clear Naming	✅ Passed	New helper/method names are specific and focused; partitionStartDate and ensurePartitions each do one clear job, and the new test matches that scope.
Feature Documentation	✅ Passed	No docs/features page covers prowloader partitioning; the only matching feature doc is about labels/symptoms and is unaffected by this internal loader fix.
Stable And Deterministic Test Names	✅ Passed	Added test titles are static descriptive strings; no Ginkgo titles or dynamic data appear in the new tests.
Microshift Test Compatibility	✅ Passed	No Ginkgo e2e tests were added; TestPartitionStartDate is a plain unit test using only time/ProwJob types, with no MicroShift-incompatible APIs.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	No new Ginkgo e2e tests were added; the new test is a standard Go unit test using testing/t.Run, with no SNO-specific assumptions.
Topology-Aware Scheduling Compatibility	✅ Passed	Only prowloader partition-range logic and unit tests changed; no manifests, controllers, or topology-dependent scheduling constraints were added.
Ote Binary Stdout Contract	✅ Passed	The PR only changes prowloader partition logic; no stdout writes were added in main/init/TestMain or suite setup, and prowloader has no such entry points.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The PR adds a Go unit test only; no new Ginkgo e2e tests, IPv4-only IP logic, or external connectivity assumptions were found.
No-Weak-Crypto	✅ Passed	The PR only changes partition-date logic and logging; no crypto primitives, custom crypto, or secret/token comparisons are added.
Container-Privileges	✅ Passed	HEAD changes only pkg/dataloader/prowloader/prow.go; no container/K8s manifest files were modified, so no privileged settings were introduced.
No-Sensitive-Data-In-Logs	✅ Passed	New ensurePartitions logs only release names and partition start dates; no passwords, tokens, PII, hostnames, or customer data are exposed.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	Clear and specific; it accurately describes the main change of extending the partition start bound for outlier job start times.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[petr-muller-reviewer]

Solid, well-targeted fix. Adapting the partition window to actual data instead of guessing a larger grace period is the right design. Code is clean, well-tested, and follows project conventions. Two nits on comment accuracy, one observability question.

[openshift-ci]

@petr-muller-reviewer: changing LGTM is restricted to collaborators

Details

In response to this:

Solid, well-targeted fix. Adapting the partition window to actual data instead of guessing a larger grace period is the right design. Code is clean, well-tested, and follows project conventions. Two nits on comment accuracy, one observability question.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[petr-muller]

/retitle TRT-2856: Extend partition start bound to cover outlier job start times

[openshift-ci-robot]

@petr-muller: This pull request references TRT-2856 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

• sippy-backup CronJob pods (fresh-DB sippy load --init-database --release=4.20) have been failing repeatedly with:

inserting prow_job_run_tests: ERROR: no partition of relation "prow_job_run_tests_p4_20" found for row (SQLSTATE 23514)

• Root cause: ensurePartitions() only creates daily partitions covering [loadSince-1day, now+2days]. Jobs that Prow eventually marks aborted after getting stuck can report a StartTime days before their completion time (the field the BigQuery fetch query actually filters on), so such a job can be fetched while its StartTime falls outside the partition window. Confirmed one such job directly: periodic-ci-openshift-release-main-nightly-4.20-e2e-agent-single-node-ipv6-conformance (build 2076094714775343104), started 2026-07-12, not marked complete (aborted) until 2026-07-15 — an 80 hour gap, well past the existing 1-day grace.
• This isn't a one-off: over the last 30 days, several aborted jobs (all releases) had start→completion gaps beyond 24h, with the worst observed around 83h.
• Extend the partition start bound to the earliest Status.StartTime actually present in the fetched job set (with the existing loadSince - 1 day as a floor), so partitions always cover the real data about to be written, regardless of how anomalous a single job's timestamp is.

This is a follow-up to #3798 (which stops one bad batch from aborting/cancelling the entire load) — this PR addresses the actual partition-range gap that causes the batch to fail in the first place.

Test plan

• go build ./pkg/dataloader/prowloader/...
• go vet ./pkg/dataloader/prowloader/...
• go test ./pkg/dataloader/prowloader/...
• golangci-lint run ./pkg/dataloader/prowloader/... (no new findings)
• Added TestPartitionStartDate covering: no jobs, jobs within the grace window, zero-value start times, and an outlier job extending the bound

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
• Improved historical job coverage by extending the partition start bound when workflow runs began earlier than the configured load window.
• Partition selection now considers actual job start times (ignoring zero/unknown values) and falls back to the existing grace-period behavior when none are available.
• Tests
• Added unit coverage for the partition-start calculation, including grace-window, missing data, and earlier-than-window scenarios.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-merge-bot]

Scheduling required tests:
/test e2e

[openshift-ci]

@petr-muller: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[mstaeble]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mstaeble, petr-muller, petr-muller-reviewer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [mstaeble,petr-muller]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment