-
Notifications
You must be signed in to change notification settings - Fork 1
feat(supply-chain): verify locked hashes against PyPI releases #1370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
seonghobae
merged 35 commits into
feat/dependency-lock-provenance-receipt
from
feat/python-lock-registry-provenance
Aug 25, 2026
Merged
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
ae5ab23
test(supply-chain): add RED lock provenance contracts
seonghobae 912be00
feat(supply-chain): implement offline Python lock provenance receipt
seonghobae 027d227
ci(supply-chain): publish Python lock provenance receipt
seonghobae c59f7d6
test(supply-chain): harden lock provenance branch coverage
seonghobae 05b2cc0
fix(supply-chain): fail closed on incomplete lock generators
seonghobae f5b75fe
docs(supply-chain): record Python lock provenance evidence boundary
seonghobae df3efe5
fix(supply-chain): fail closed when lock hashes disappear
seonghobae 3a7acca
test(supply-chain): cover provenance review regressions
seonghobae 59aa696
fix(supply-chain): contain lock provenance reads
seonghobae 2f245bb
fix(ci): publish failed lock provenance receipts
seonghobae 1b99119
docs(supply-chain): document contained provenance reads
seonghobae dcd0e35
test(supply-chain): specify registry hash provenance contract
seonghobae f816166
feat(supply-chain): validate locked hashes against PyPI releases
seonghobae d9dfedf
ci(supply-chain): verify PyPI hashes before install
seonghobae cfcf064
docs(supply-chain): record PyPI hash provenance boundary
seonghobae fa6063b
test(supply-chain): cover PyPI provenance failure boundaries
seonghobae 3dbdf3b
test(supply-chain): reject PyPI metadata origin redirects
seonghobae 3ff0bf9
fix(supply-chain): keep PyPI metadata reads on trusted origin
seonghobae 712e8e1
test(supply-chain): keep registry edge suite lint-clean
seonghobae ebb91a7
test(supply-chain): reject vacuous PyPI provenance receipts
seonghobae ecb492c
fix(supply-chain): require non-vacuous registry evidence
seonghobae 6fdb12d
test(supply-chain): expose recursive requirements include bypass
seonghobae 65f5a88
fix(supply-chain): validate recursive requirements includes
seonghobae bb8e349
docs(supply-chain): record recursive include boundary
seonghobae 80454ee
Merge live provenance parent into registry slice
seonghobae f9f7f6a
Merge branch 'develop' into feat/dependency-lock-provenance-receipt
seonghobae d6c8030
Merge branch 'feat/dependency-lock-provenance-receipt' into feat/pyth…
seonghobae 18cb855
Merge branch 'develop' into feat/dependency-lock-provenance-receipt
seonghobae 4313e25
Merge branch 'develop' into feat/dependency-lock-provenance-receipt
opencode-agent[bot] 20f1a5e
Merge branch 'feat/dependency-lock-provenance-receipt' into feat/pyth…
opencode-agent[bot] 4ad12b9
fix(ci): reject unvalidated PyPI redirects
seonghobae 1a6ac60
Merge remote-tracking branch 'origin/feat/python-lock-registry-proven…
seonghobae e5e99b4
fix(http): reject explicit zero loopback ports (#1337)
seonghobae 4535698
merge(develop): reconcile supply-chain hash verification with lock pr…
seonghobae a1f89eb
merge(supply-chain): adopt evolved lock provenance attestation into r…
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| """Non-vacuous evidence contracts for PyPI lock provenance.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import importlib.util | ||
| import json | ||
| import runpy | ||
| import sys | ||
| import urllib.request | ||
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| import pytest | ||
|
|
||
| REPO_ROOT = Path(__file__).resolve().parents[2] | ||
| SCRIPT_PATH = REPO_ROOT / "scripts" / "ci" / "python_lock_registry_provenance.py" | ||
| _spec = importlib.util.spec_from_file_location("python_lock_registry_non_vacuous", SCRIPT_PATH) | ||
| assert _spec is not None and _spec.loader is not None | ||
| registry_provenance = importlib.util.module_from_spec(_spec) | ||
| sys.modules[_spec.name] = registry_provenance | ||
| _spec.loader.exec_module(registry_provenance) | ||
|
|
||
|
|
||
| def _sha() -> str: | ||
| """Return one deterministic SHA-256 fixture digest.""" | ||
| return "a" * 64 | ||
|
|
||
|
|
||
| def _codes(receipt: dict[str, object]) -> set[str]: | ||
| """Return stable top-level violation codes from a repository receipt.""" | ||
| return {str(item["code"]) for item in receipt["violations"]} | ||
|
|
||
|
|
||
| def test_repository_without_hash_locks_fails_non_vacuously(tmp_path: Path) -> None: | ||
| """A green registry receipt must represent at least one discovered hash lock.""" | ||
| (tmp_path / "requirements.txt").write_text("example==1.0\n", encoding="utf-8") | ||
|
|
||
| receipt = registry_provenance.validate_repository_registry( | ||
| tmp_path, | ||
| fetch_release=lambda project, version: {}, | ||
| ) | ||
|
|
||
| assert receipt["status"] == "failed" | ||
| assert receipt["lock_files"] == [] | ||
| assert _codes(receipt) == {"registry-no-hash-locks"} | ||
|
|
||
|
|
||
| class _Response: | ||
| """Minimal exact-origin PyPI response used by the script-entrypoint test.""" | ||
|
|
||
| def __init__(self, url: str) -> None: | ||
| self.url = url | ||
| self.headers = {"Content-Type": "application/json"} | ||
| self.payload = json.dumps( | ||
| { | ||
| "info": {"name": "example", "version": "1.0"}, | ||
| "urls": [ | ||
| { | ||
| "packagetype": "sdist", | ||
| "yanked": False, | ||
| "digests": {"sha256": _sha()}, | ||
| } | ||
| ], | ||
| } | ||
| ).encode("utf-8") | ||
|
|
||
| def __enter__(self) -> "_Response": | ||
| return self | ||
|
|
||
| def __exit__(self, *args: Any) -> None: | ||
| return None | ||
|
|
||
| def geturl(self) -> str: | ||
| """Return the unchanged trusted request URL.""" | ||
| return self.url | ||
|
|
||
| def read(self, size: int) -> bytes: | ||
| """Return a bounded response body.""" | ||
| return self.payload[:size] | ||
|
|
||
|
|
||
| def test_script_main_guard_runs_registry_validation( | ||
| tmp_path: Path, | ||
| monkeypatch: pytest.MonkeyPatch, | ||
| capsys: pytest.CaptureFixture[str], | ||
| ) -> None: | ||
| """Executing the script as __main__ publishes a passing JSON receipt and exits zero.""" | ||
| lock = tmp_path / "requirements-hashes.txt" | ||
| lock.write_text( | ||
| f"example==1.0 \\\n --hash=sha256:{_sha()}\n", | ||
| encoding="utf-8", | ||
| ) | ||
|
|
||
| class _Opener: | ||
| def open(self, request: urllib.request.Request, timeout: float) -> _Response: | ||
| return _Response(request.full_url) | ||
|
|
||
| monkeypatch.setattr(urllib.request, "build_opener", lambda *handlers: _Opener()) | ||
| monkeypatch.setattr( | ||
| sys, | ||
| "argv", | ||
| [ | ||
| str(SCRIPT_PATH), | ||
| "--repository-root", | ||
| str(tmp_path), | ||
| "--json", | ||
| ], | ||
| ) | ||
|
|
||
| with pytest.raises(SystemExit) as exit_info: | ||
| runpy.run_path(str(SCRIPT_PATH), run_name="__main__") | ||
|
|
||
| assert exit_info.value.code == 0 | ||
| assert json.loads(capsys.readouterr().out)["status"] == "passed" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 New CI gate makes backend job depend on live PyPI reachability for ~120 pinned packages with no retry
The new step (
\.github/workflows/app-ci.yml:62-73) runspython_lock_registry_provenance.py, which discovers everyrequirements*.txthash lock in the repo (backend/requirements-hashes.txt ~105 pins, backend/requirements-agent.txt, connector/requirements-hashes.txt, requirements-strix-ci-hashes.txt, requirements-bandit-ci-hashes.txt) and issues one liveGET pypi.org/pypi/<name>/<ver>/jsonper unique (project,version).fetch_pypi_release(python_lock_registry_provenance.py) makes a single attempt with a 15s timeout and no retry;cached_fetch(python_lock_registry_provenance.py) caches failures so a single transient 5xx/timeout on any one of ~120 sequential requests permanently emitsregistry-metadata-fetch-failedand fails the whole backend job (nocontinue-on-error). The docs describe the gate as intentionally fail-closed, but conflating a transient network error with a provenance failure is a real CI-flakiness source that runs on every PR to develop/master and every push. Consider bounded retries/backoff for transport errors distinct from genuine provenance mismatches.Was this helpful? React with 👍 or 👎 to provide feedback.