fix(gs): decode sceGsSetDefDBuff trailing args via EABI registers, not o32 stack offsets - #186
Draft
smmathews wants to merge 1 commit into
Draft
fix(gs): decode sceGsSetDefDBuff trailing args via EABI registers, not o32 stack offsets#186smmathews wants to merge 1 commit into
smmathews wants to merge 1 commit into
Conversation
…t o32 stack offsets sceGsSetDefDBuff read ztest/zpsm/clear with readStackU32 at offsets 16/20/24. Under the EABI this runtime targets, arguments 5-8 arrive in $t0-$t3 (GPRs 8-11), so those stack offsets hold caller-frame bytes, not the trailing args. The sibling sceGsSetDefDBuffDc already decodes the same three args correctly with decodeGsTrailingArgs3 (registers first, stack only as an all-zero fallback); this routes sceGsSetDefDBuff through the same helper. Adds a register-ABI regression test that seeds distinct register and stack sentinels and asserts the written ZBUF/TEST fields reflect the register values.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
sceGsSetDefDBuffreads its ztest, zpsm and clear arguments from stack offsets 16, 20and 24. Under this runtime's EABI those three are register-passed in
$t0–$t2(GPRs 8–10), so the reads pick up caller-frame bytes and seed the double buffer's Z-test
and Z-format from stale stack data on every call.
Fix
Swap the three
readStackU32calls fordecodeGsTrailingArgs3, which reads the registersfirst and falls back to the old stack offsets only when all three registers are zero.
That helper already exists and is already used by
sceGsSetDefDBuffDcfor the identicalthree arguments — this makes the two siblings consistent. Because the fallback triggers
only on an all-zero register triple, the sole behavioural change is for register-passed
nonzero trailing arguments, which is the buggy case.
Basis
The EE calling convention passes eight integer arguments in GPRs 4–11. The test for any
readStackU32site is how many register arguments precede it: fewer than eight is a bug,eight or more is correct.
sceGifPkRefLoadImagein the same file also callsreadStackU32, at offsets 0 and 8, andis left untouched — it consumes eight register arguments first (GPRs 4–11), so its width
and height genuinely are the ninth and tenth arguments and belong on the stack.
#195 fixes the same defect class in
sceSifSendCmd.Testing
One added test,
sceGsSetDefDBuff decodes trailing args from the recompiler register ABI:it seeds distinct sentinels in GPRs 8–10 and at the o32 stack offsets, then asserts the
written ZBUF and TEST fields carry the register values rather than the stack bytes.
The
-include cstdintis needed to build this branch as it stands: the vendored ELFIOheader omits it and newer libstdc++ no longer supplies the typedefs transitively. #199
fixed that on
main, so the flag becomes unnecessary once this branch is rebased.Mutation: revert the three-line helper swap and the two new assertions report the stack
sentinels instead of the register sentinels.
Risk and not in scope
readStackU32sites was done.SIF.cppandMemoryCard.cppalso call it; auditing them is separate follow-up work.genuinely pass zeros in all three registers, so such a caller sees no change.