Skip to content

CMP-4409: Add per-package coverage tracking to make test-unit#1289

Open
rhmdnd wants to merge 1 commit into
ComplianceAsCode:masterfrom
rhmdnd:add-coverage-tracking
Open

CMP-4409: Add per-package coverage tracking to make test-unit#1289
rhmdnd wants to merge 1 commit into
ComplianceAsCode:masterfrom
rhmdnd:add-coverage-tracking

Conversation

@rhmdnd

@rhmdnd rhmdnd commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Adds a coverage ratchet to the unit test workflow: make test-unit now
runs with -cover, captures native go test per-package coverage output,
and compares it against a committed baseline. The coverage check fails
if any package regresses below the baseline or if coverage has improved
and the baseline is out of date, ensuring it stays current.

New files:

  • hack/check-coverage.sh: parses go test output (plain and JSON
    formats) and compares per-package coverage against the baseline
  • hack/update-coverage-baseline.sh: generates the baseline from
    go test output
  • coverage-baseline.txt: committed baseline with current per-package
    coverage percentages

Makefile changes:

  • test-unit captures output via tee and runs check-coverage if a
    baseline exists
  • New update-coverage-baseline and check-coverage targets
  • TESTABLE_PKGS filters to packages with test files to avoid
    go 1.25 covdata tool errors on empty packages

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

@openshift-ci openshift-ci Bot requested review from Vincent056 and mrogers950 June 29, 2026 18:30
@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rhmdnd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rhmdnd rhmdnd changed the title Add per-package coverage tracking to make test-unit CMP-4409: Add per-package coverage tracking to make test-unit Jun 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Collaborator

@rhmdnd: This pull request references CMP-4409 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Adds a coverage ratchet to the unit test workflow: make test-unit now
runs with -cover, captures native go test per-package coverage output,
and compares it against a committed baseline. The coverage check fails
if any package regresses below the baseline or if coverage has improved
and the baseline is out of date, ensuring it stays current.

New files:

  • hack/check-coverage.sh: parses go test output (plain and JSON
    formats) and compares per-package coverage against the baseline
  • hack/update-coverage-baseline.sh: generates the baseline from
    go test output
  • coverage-baseline.txt: committed baseline with current per-package
    coverage percentages

Makefile changes:

  • test-unit captures output via tee and runs check-coverage if a
    baseline exists
  • New update-coverage-baseline and check-coverage targets
  • TESTABLE_PKGS filters to packages with test files to avoid
    go 1.25 covdata tool errors on empty packages

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@rhmdnd rhmdnd requested review from taimurhafeez and removed request for mrogers950 June 29, 2026 18:32
@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1289-657c6df5507bd28ac74789b4e2ff1f9ff5c38293

Comment thread hack/update-coverage-baseline.sh Outdated
@rhmdnd rhmdnd force-pushed the add-coverage-tracking branch from 657c6df to cf990f9 Compare July 1, 2026 10:42
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1289-cf990f9323019248e8d1e110641ec7d4d354e560

@openshift-ci

openshift-ci Bot commented Jul 1, 2026

Copy link
Copy Markdown

@rhmdnd: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-rosa 657c6df link true /test e2e-rosa
ci/prow/e2e-aws-parallel-arm 657c6df link true /test e2e-aws-parallel-arm
ci/prow/e2e-aws-serial-arm 657c6df link true /test e2e-aws-serial-arm
ci/prow/e2e-aws-deployment 657c6df link true /test e2e-aws-deployment

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Comment thread Makefile Outdated
Comment thread Makefile Outdated
Adds a coverage ratchet to the unit test workflow: make test-unit now
runs with -cover, captures native go test per-package coverage output,
and compares it against a committed baseline. The coverage check fails
if any package regresses below the baseline or if coverage has improved
and the baseline is out of date, ensuring it stays current.

New files:
- hack/check-coverage.sh: parses go test output (plain and JSON
  formats) and compares per-package coverage against the baseline
- hack/update-coverage-baseline.sh: generates the baseline from
  go test output
- coverage-baseline.txt: committed baseline with current per-package
  coverage percentages

Makefile changes:
- test-unit captures output via tee and runs check-coverage if a
  baseline exists
- New update-coverage-baseline and check-coverage targets
- TESTABLE_PKGS filters to packages with test files to avoid
  go 1.25 covdata tool errors on empty packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rhmdnd rhmdnd force-pushed the add-coverage-tracking branch from cf990f9 to e99d22f Compare July 10, 2026 11:28
@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1289-e99d22fe328f6706ecfaf5082f11764f9035c4b7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants