Repository navigation
release: orchestrate signed Python driver releases - #1175
Conversation
Pin stable release metadata and retain the tested source tree before publication. Preserve signed history and distribution bytes so retries can verify existing uploads and continue without moving tags or signing published files again. Add public-only signature verification, isolated signing-key cleanup and helper coverage for failure, cancellation and partial-upload recovery. Run portable metadata guards in the primary unit suite as well.
d6994c1 to
fc52a4b
Compare
Replace tag-triggered uploads with a manual master-only release flow. Validate the bundled candidate across the existing wheel targets and the pinned integration matrix before signing and publication. Run complete dry runs with signing secrets and no approval gate. Keep real releases behind approval, publish through PyPI OIDC, and deploy docs after the retained history and immutable GitHub release are verified.
fc52a4b to
bc64542
Compare
Run preparation, builds and validation before the protected environment. Keep dry-run signing approval-free without publisher or OIDC credentials. On failed-job reruns, recheck the current run signed checkpoint before signing so saved history and signatures remain unchanged.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1175 +/- ##
==========================================
+ Coverage 87.36% 87.39% +0.02%
==========================================
Files 44 44
Lines 19272 19272
Branches 3015 3015
==========================================
+ Hits 16837 16842 +5
+ Misses 1783 1780 -3
+ Partials 652 650 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Code Review by Qodo
🔴 High 1. Release stops for write-only publishers
|
| api("immutable-releases")["enabled"] is True, | ||
| "Enable repository release immutability before publication.", | ||
| ) |
There was a problem hiding this comment.
🔴 High
1. Release stops for write-only publishers 🐞 Bug ≡ Correctness
prepublish() reads the repository's immutable-release setting using PUBLISHER_TOKEN, but that endpoint requires administration read access. If the publisher receives only the documented repository write access, preflight fails before the PyPI upload even after the bypass team is configured.
Agent Prompt
## Issue description
The immutable-release settings endpoint requires administration read access, while the documented publisher setup grants repository write access only. That configuration blocks real releases at preflight.
## Fix Focus Areas
- scripts/release/release.py[788-796]
- scripts/release/README.md[97-106]
- .github/workflows/release.yml[244-248]
## Recommended Fix
Ensure the credential used for this preflight has the required administration read access without broadening the publisher bypass team's membership; document and verify the required credential permissions during cutover.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Issue
Release tags currently upload to PyPI directly from the build workflow. This leaves metadata preparation, signing, integration validation and recovery outside a single approved release flow.
Fix
Add a manual
release.ymlflow modeled on the Java driver release process. It prepares metadata from a pinnedmastercommit, exports the candidate as a Git bundle, and validates the existing wheel coverage plus a pinnedpython-driver-matrixrunner across four Scylla aliases and protocols 3/4.One approval-protected
publishjob signs and publishes the real release. It creates a release commit with exactly the tested tree, a signed annotated tag, distribution signatures and a checksum manifest. An immutable artifact checkpoint retains the original packages and signed history before the top-level PyPI OIDC job uploads missing files. Publication verifies PyPI hashes, atomically pushes the retained history asscylladb-publisher, completes an immutable GitHub release, and then calls docs deployment. Existing build workflows remain available with their PyPI upload removed.Decisions
X.Y.Zversions frommaster, retainingscylla-driverandX.Y.Z-scyllatags. Keep the package at its released version afterward.dry-runbuilds, tests, signs and verifies without publication or environment approval. Itsmaster-onlyrelease-dry-runenvironment has signing secrets only; real releases retain approval. Full test coverage is preserved.skip-testsaffects project tests only; helper tests always run and the external matrix has a separate skip input.Validation
git diff --checkpassed.Cutover
Already configured:
releaseenvironment withscylladb/driversapproval, self-review allowed, administrator bypass disabled andmasteronly; signing secrets and publisher credential loaded through stdin; public certificate registered onscylladb-publisher; repository release immutability enabled;release-dry-runconfigured without reviewers, restricted tomaster, containing only the two signing secrets.Still required before the first real release:
admin:org; publication checks this before any PyPI upload.scylladb/python-driver,release.yml, environmentrelease, and retire the old upload-workflow registrations at cutover. No authenticated PyPI management connection was available.masterwith both test skips disabled. The first real release exercises PyPI OIDC.The release runbook documents these steps and retained-state recovery. This changes release automation without changing the driver runtime or protocol behavior.
Pre-review checklist
./docs/source/.Fixes:annotations to PR description.