blacklist the __pypy__ helpers that attack the fuzzer instead of the target - #257
Merged
Conversation
…target
Caught on the first PyPy 3.11 fleet config. Five members of __pypy__ are debug/test
plumbing, not fuzzable surface, and each manufactures a crash that never happened:
- attach_gdb: runs an interp-level gdb *inside the session*. Its banner ("For bug
reporting instructions...") lands in the captured stdout and scores on the "bug" word.
Observed live: session kept as `__pypy__-bug-systemerror`.
- _internal_crash: documented as "for testing purposes, raise an interpreter-level
ValueError. Should turn into a SystemError automatically". SystemError is a 1.0 crash
word, so with --test-private this alone tagged 4 of 4 __pypy__ sessions as crashes.
- remote_exec: "Executes a script of Python code in a given remote Python process" -- a
fuzzer-chosen pid is arbitrary code injection into any process on the box, including
sibling fleet instances and the fuzzer itself.
- set_code_callback: process-global hook run on every code-object creation; handed one of
fusil's bomb objects, the rest of the session detonates on unrelated code.
- pyos_inputhook / revdb_stop: block.
The entry is deliberately narrow -- __pypy__ is PyPy-only surface with no CPython
counterpart, which is the reason to fuzz PyPy at all, so newdict/strategy/internal_repr/
intop/move_to_end and the rest stay fuzzable. A test pins both directions.
Verified against PyPy 3.11.15 / 7.3.23: 5 of 5 __pypy__ sessions now come back clean
(previously 4 of 4 false-tagged) with 3.3k-4.9k lines of real fuzzing each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WhcpLoyjUWLbETGZnA9boj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #256, caught while validating the first PyPy 3.11 fleet config.
Five members of
__pypy__are debug/test plumbing rather than fuzzable surface, and eachmanufactures a crash that never happened:
attach_gdbFor bug reporting instructions...) hits thebugword. Observed live as__pypy__-bug-systemerror._internal_crashSystemErroris a 1.0 crash word — with--test-privatethis alone tagged 4 of 4__pypy__sessions as crashesremote_execset_code_callbackpyos_inputhook,revdb_stopPyOS_InputHook()/ reverse-debugger stopThis is the "generated code kills its own session" class from #192, plus one genuine host
hazard (
remote_exec).Deliberately narrow.
__pypy__is PyPy-only surface with no CPython counterpart —the reason to fuzz PyPy at all — so
newdict/strategy/internal_repr/intop/move_to_endand the rest stay fuzzable.test_pypy_blacklist_stays_narrowpins thatdirection so a future edit can't quietly blanket the module.
Verification
PyPy 3.11.15 / 7.3.23,
--test-private --functions-number 60:__pypy__-systemerrorfuzzing per session
Full suite green (1249 tests);
ruff check+ruff format --checkclean.🤖 Generated with Claude Code