Skip to content

Pilot an automated contribution gate, and gate e2e tests on lint - #8453

Open
akhenry wants to merge 8 commits into
masterfrom
contribution-workflow
Open

akhenry wants to merge 8 commits into
masterfrom
contribution-workflow

Conversation

@akhenry

@akhenry akhenry commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

No single issue: this implements the new rules and process for contributions.

Describe your changes:

Three commits, each reviewable on its own.

1. [CI] Run end-to-end tests only once lint passes. This applies to every pull request.

  • Every e2e job, and the unit tests, now wait for lint, so a style error costs one short run instead of an hour of browser time.
  • e2e-couchdb moves into pr.yml so it can wait for e2e-ci. Its Codecov upload no longer fails the job, because forks get no Codecov token, so it failed every fork PR.
  • The two visual-a11y-ci shards get their own names, so a failing shard can't hide behind a passing one.
  • PRs re-run CI on synchronize, reopened and ready_for_review. Fork PRs never re-ran CI on new commits before. Pushes now only run CI on master and release/*.

2. [Contribution Gate] Pilot an automated contribution gate. This applies only to PRs labelled new-workflow-candidate.

A bot takes a candidate PR through the contribution rules, then the required checks, then any CodeQL findings (which need a reply), and only then requests a team review. It keeps one comment up to date, sets a 7-day deadline (30 for drafts), sends one reminder, and closes the PR when the deadline passes. /recheck reopens it once it's fixed. Team PRs are never closed.

It runs alongside PR Cop and the current templates, and nothing else changes:

  • Unlabelled PRs: the workflow job is skipped before it starts, and the gate checks the label again itself.
  • Closing: nothing outside the pilot can ever be closed. The daily sweep only looks at labelled contributions.
  • Templates: it reads the current ones as they are. It accepts the enhancement template's own headings, and never asks for the type-label or milestone checklist items, because it sets those itself.
  • Check Milestone: branch protection requires it. The gate publishes its own copy beside PR Cop's, using the same rule, and never edits PR Cop's job check.

The AI review stage is written but switched off until an AI reviewer works. Copilot silently drops review requests from accounts without entitlement, so the gate reads every request back and fails open. The README covers the pilot, the AI stage, and a switch-over checklist.

3. [Documentation] Clarify what contributions are accepted, and how.

  • Enhancements need a maintainer's approval on the issue before work starts.
  • Vulnerability reports need a reproduction.

How to try it

Create the new-workflow-candidate label, then add it to a PR. The gate starts straight away and posts its comment. Every action it takes appears in the workflow log as a line starting Gate:.

Testing done

  • npm run test:contribution-gate: 68 tests against a stubbed GitHub API. They cover the rules, the clock, the reminder, closing, /recheck reopening, the pilot scope, the current templates (read from the repo's own template files), and check ownership.
  • Run live on a fork, end to end:
    • A non-compliant PR got its comment and deadline. Made compliant, it moved to awaiting CI, then team review once CI passed.
    • Reminder, close, /recheck while still incomplete (stays closed), and /recheck after a fix (reopens).
    • An unlabelled PR was skipped. Labelling it started the gate.
  • Bugs that live testing caught and fixed here:
    • A closed PR is frozen at its closing commit, which deadlocked /recheck. A closed PR is now judged by its branch tip.
    • The gate would have rewritten PR Cop's Check Milestone job check.

Not verified until this is on nasa

  • Requesting the openmct-maintainers team. If the request is dropped, the gate falls back to an @-mention comment.
  • That branch protection reads the gate's newer Check Milestone rather than PR Cop's. If it doesn't, any edit to the PR description re-runs PR Cop, which then passes.

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes?
  • Has this been smoke tested?
  • Have you associated this PR with a type: label? Note: this is not necessarily the same as the original issue.
  • Have you associated a milestone with this PR? Note: leave blank if unsure.
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Lint is cheap and fails fast, so every end-to-end job, and the unit
tests, now wait for it instead of burning browser minutes on code that
already needs another push.

- e2e-couchdb moves into pr.yml so it can wait for e2e-ci: it tests the
  same application against a real database, and is the most expensive
  job we run. Its Codecov upload no longer fails the job, which broke it
  for every pull request from a fork, since forks get no Codecov token.
- perf-test and mem-test drop if: always(), which would have run them
  even when lint failed.
- The two visual-a11y shards get their own names, so a failing shard
  can no longer hide behind a passing one under a shared check name.
- Pull requests re-run CI on synchronize, reopened and ready_for_review,
  which fork pull requests never did before. Pushes only run CI on
  master and release branches, since pull requests now cover the rest.
Adds a gate that takes a pull request through the contribution rules,
the required checks and any CodeQL findings before the maintainers are
asked to look, with one kept-up-to-date comment, a deadline, a single
reminder, and /recheck to reopen once the points are fixed.

It is a pilot. It runs alongside PR Cop and the current templates, and
acts only on issues and pull requests a maintainer has labelled
new-workflow-candidate, so nothing else changes and nothing outside the
pilot is ever closed. Adding the label is what starts it.

So that it can run alongside the current process, the gate reads the
current templates as they are: the enhancement template's own headings
count, and it never asks a contributor to tick the checklist items about
the type label or milestone, because it sets those itself. It also
publishes the required Check Milestone check by PR Cop's rule, because a
label it adds would not re-run PR Cop.

A closed pull request is judged by the tip of its branch, so that a fix
pushed after closing can reopen it. GitHub freezes a closed pull
request at the commit it was closed on.

The AI review stage is written but switched off until an AI reviewer
works. The README covers the pilot, and what switching over involves.
Enhancements and new functionality need a maintainer's approval on the
issue before work starts, because Open MCT is used by flight controllers
and core should stay lightweight; external plugins are encouraged as the
alternative. Bug fixes can go straight to a pull request.

Vulnerability reports must now include a reproduction, either manual
exploit steps or an end-to-end test of a real-world workflow. Hardening
suggestions without an exploitable vulnerability should be filed as bugs.
current = previous.replace(HTML_COMMENT_PATTERN, '');
} while (current !== previous);

return current.replace(/<!--/g, '').replace(/-->/g, '');
current = previous.replace(HTML_COMMENT_PATTERN, '');
} while (current !== previous);

return current.replace(/<!--/g, '').replace(/-->/g, '');
@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.90%. Comparing base (e09bfaf) to head (3ddf450).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8453   +/-   ##
=======================================
  Coverage   56.90%   56.90%           
=======================================
  Files         731      731           
  Lines       30057    30057           
  Branches     2857     2857           
=======================================
+ Hits        17103    17105    +2     
+ Misses      12622    12620    -2     
  Partials      332      332           
Flag Coverage Δ *Carryforward flag
e2e-ci 65.30% <ø> (+<0.01%) ⬆️
e2e-couchdb 39.23% <ø> (?)
e2e-full 39.20% <ø> (ø) Carriedforward from a6c012d
unit 45.75% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.
see 2 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e09bfaf...3ddf450. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

akhenry added a commit that referenced this pull request Oct 6, 2026
The contribution gate, the CI changes and the contribution policy moved
to #8453, so this branch is about the LAD clock alone. Every
workflow and process file goes back to master's version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akhenry
akhenry force-pushed the contribution-workflow branch from f1ebc3a to d4285d5 Compare October 7, 2026 00:06
Commits made with Claude Code end with "Made with Claude" rather than a
Co-Authored-By trailer. .claude/settings.json is the shared project
settings file, so it is committed; everything else under .claude/ stays
local.
Empty strings rather than false, because Claude Code versions before
2.1.281 reject false and would skip the whole project settings file.
@akhenry
akhenry force-pushed the contribution-workflow branch from 0fe9c58 to 5a19ad5 Compare October 7, 2026 03:49
akhenry added a commit that referenced this pull request Oct 7, 2026
The contribution gate, the CI changes and the contribution policy moved
to #8453, so this branch is about the LAD clock alone. Every
workflow and process file goes back to master's version.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants