fix(agc): return 0 for out-of-range sceAgcInit without table lookup

[mazen320]

What

Pointer-shaped / out-of-range sceAgcInit values (>=14) return 0 without a register-default lookup. Versions 0–13 still select their tables. sceAgcInit_0090 checks the state pointer and then calls sceAgcInit, as OpenAGC does.

#4808 console probe (firmware 13.60): sceAgcInit(&buf, 8) (NID kW3GLb7QfPg) returns 0 and writes nothing. The probe does not establish that table 13 is selected. Scalar versions 2–13 return 0x8A6C0004 on that console but still return 0 here (documented in debt; not changed by this PR).

Tested

• Windows 11, WinLibs MinGW-w64 GCC 15.2.0 r7, Release, BUILD_TESTING=ON
• ctest --test-dir build-win -R 'agc_initialization|agc_commands' --output-on-failure — 2/2 passed
• python -I tools/check_conventions.py --base upstream/main --head HEAD — passed
• Titles not rerun on this revision; earlier Windows/AMD title evidence on this PR still applies to returning 0 for the pointer-shaped call

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: PPSA99011 sceAgcInit throws for an out-of-range version; PS5 FSR4 Showcase calls it with a pointer and expects an error result #4808; PPSA99995 (same stop)
• AI-assisted: yes

[MarcZX10]

I checked the call path and the GetRegisterDefaults results are used only to initialize the cached table objects; sceAgcInit does not store a selected version or return that table. The two reported firmware runs therefore support returning success for these pointer-shaped calls, while they do not establish which table firmware selects. The PR description and debt entry preserve that distinction, so I found no blocking issue in this diff.

[coltken95]

Tested on Linux x86-64 (Release build), with 0cc5102 merged into main 9d0a41d:

• With the PR: agc_initialization and agc_commands pass.
• With main's sources and the PR's tests: both fail.

AI-assisted: yes (Claude Code)

[LeandroLP23]

Title runs on Windows with AMD, since the description says the titles were not rerun. Setup: Windows 11 (26300), Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), WinLibs GCC 15.2.0, Release. 0cc51023b merged onto main a0f5916a2. Catalog builds, 40 s runs. PPSA99011 was relinked with --trap-sdk-syscall-stubs.

agc_initialization and agc_commands pass.

Title	main	main + this PR (3 runs)
PS5 FSR4 Showcase (PPSA99011 01.000.010)	Exits after 4 s. Its log: FSR4_SHOWCASE_ERROR vkCreateDevice(physical, &di, NULL, &device) = -3, result=FAIL	Creates the device and its pipelines, then logs FSR4_SHOWCASE_READY version=01.000.010 display=1920x1080. Stops 7-10 s in at the next, unrelated limit (below)
Prospero Win (PPSA99995 0.2.0)	Alive at 40 s, but its log reports launcher video=state and no window appears	Logs launcher video=ok and presents: window Prospero Win | FPS: 1.02 at 40 s, 3 of 3 runs

The next stop of PPSA99011 with this PR is AGC driver: COPY_DATA at DWORD 69: COPY_DATA register, GDS or reference-clock source is not implemented. These are three COPY_DATA packets (control 0x00100200) that copy the SH registers 0x2e0c, 0x2e07 and 0x2e00 (COMPUTE_PGM_LO, COMPUTE_NUM_THREAD_X, COMPUTE_DISPATCH_INITIATOR) to memory, next to a DISPATCH_DIRECT.

AI-assisted: yes (Claude Code)

[MarcZX10]

Thanks for the Windows/AMD follow-up. This adds useful title-level evidence: PPSA99011 gets past device creation and reaches the later COPY_DATA stop, while Prospero Win opens and presents in all three runs. The next FSR4 stop is now covered separately by #5477, which implements the register-to-memory copies and tests the dispatch register values. The table-selection behavior above version 13 remains unverified, as the PR description states.

[oneandonlydean]

The console measurement is in #4808 now (#4808 (comment)): sceAgcInit(&buf, 8) returns 0 and writes nothing. So return 0 for these calls without picking table 13 for a stack address; choosing a table the firmware is not shown to use is a fallback. Please cite the measurement in the debt entry and the description, and note there that scalar values 2 to 13 return 0x8A6C0004 on the console but 0 on main and here.

[MarcZX10]

The PS5 probe linked from #4808 changes the evidence for this fallback: sceAgcInit(&buf, 8) returns 0 and leaves the buffer unchanged, while every tested scalar value >=2 returns 0x8A6C0004. Selecting table 13 for a pointer-shaped argument is therefore unsupported. Please make the pointer-shaped path return success without a register-default lookup, cite the probe in this PR and the debt entry, and document that scalar 2–13 still differ from the current AnyPS5 behavior. The probe does not establish that table 13 is selected or that scalar 2–13 should use those tables.

[xXPBSXx]

Windows test of #5163: Windows 10 (19045), MinGW-w64 GCC 15.2, Release, ninja -j 4, ctest called directly. Tree: main feddf7f4c + merge of PR head 0cc51023b130ddb251198b05c4a8d6eee4044b71 (local merge 6429dab37).
agc_commands (built from libSceAgc/tests/Commands.cpp): exit code 0 in 3 of 3 runs (0.05 s each). The test selects no GPU.
Not tested: that the new test case fails without the change (no baseline run), other ctest targets, Linux, a title run (PPSA99011 not available here). The PR state was UNSTABLE when I looked (CI not finished); I did not look at which check.

AI-assisted (Claude Code).

[MarcZX10]

Thanks for the controlled Windows run: agc_commands passes 3/3 on the merged PR head, which confirms the suite is stable on that setup. The new report does not cover the console-observed pointer-shaped sceAgcInit(&buf, 8) behavior, and the current diff still maps that value to table 13. The firmware evidence in #4808 remains the unresolved point in my review; please update the pointer-shaped path and cite the measurement before I can approve this revision.

[mazen320]

Linux x86-64 (WSL2 Ubuntu 24.04, GCC 13.3.0), Release, BUILD_TESTING=ON, PR head eb1b9eca9:

• ctest --test-dir build-linux -R 'agc_initialization|agc_commands' --output-on-failure — 2/2 passed
• python3 -I tools/check_conventions.py --base upstream/main --head HEAD — passed (also on Windows WinLibs 15.2.0 r7 for the same head)

AI-assisted: yes

[mazen320]

Closing temporarily to free the first-contributor open-PR slot so I can open a RetroArch Windows bring-up fix (select on regular files + AGC 8_8 UINT). Will reopen this AGC init fallback PR after that one merges (or sooner if preferred).

[mazen320]

Reopened — keeping this PR open to wait for merge. The RetroArch select + 8_8 UINT fix is parked on mazen320:fix/retroarch-windows-select-and-8_8-uint / #5795 and will be reopened after this lands.

[mazen320]

Re the overlap bot vs #5845: Depends on: none.

• This PR only changes sceAgcInit / sceAgcInit_0090 behavior + its debt bullet + tests.
• feat(libSceAgc): implement sceAgcGetDefaultCxStateFlat #5845 adds sceAgcGetDefaultCxStateFlat and inserts a new debt bullet above the sceAgcInit line; it does not implement the same export.
• The shared file is only docs/dev/debt/libSceAgc.md (adjacent lines). Whoever merges second rebases and keeps both bullets — no functional dependency either way.

[mazen320]

Triple-check follow-up (trial merges on current upstream/main 066b58fa1):

Order	Commands.cpp	Init.cpp	docs/dev/debt/libSceAgc.md
#5163 then #5845	clean auto-merge	only #5163	content conflict (adjacent bullets)
#5845 then #5163	clean auto-merge	only #5163	content conflict (adjacent bullets)

• Same exports: none (this PR: sceAgcInit / sceAgcInit_0090; feat(libSceAgc): implement sceAgcGetDefaultCxStateFlat #5845: sceAgcGetDefaultCxStateFlat).
• Not a functional dependency either way → Depends on: none still correct.
• Second merger must rebase the debt file and keep both bullets (new CxStateFlat line + updated sceAgcInit line). That is the whole overlap the bot reported.

[babyhuey]

Windows 10 evidence for this PR: merge 0d69a9e (this branch onto main 066b58f), MinGW-w64, 0 build warnings.

• agc_initialization and agc_commands: 5/5 each.
• Prospero Win (PPSA99995), 2 runs each. Main: launcher video=state, no window. Merge: launcher video=ok, a window presenting at "FPS: 1.01 (55)" and "1.02 (55)", still running at 60 s with no throws.
• PS5 FSR4 Showcase (PPSA99011, relinked with --trap-sdk-syscall-stubs and --registry). Main: FSR4_SHOWCASE_ERROR vkCreateDevice(...) = -3, then FSR4_SHOWCASE_END result=FAIL, no window. Merge: window opens, passes PIPELINES and OUTPUT, prints FSR4_SHOWCASE_READY version=01.000.010 display=1920x1080, then stops with AGC driver: COPY_DATA at DWORD 69: COPY_DATA register, GDS or reference-clock source is not implemented. That next stop is the one fix(agc): copy a shader, context or user config register to memory with COPY_DATA #5477 addresses.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#5889	core/libs/prx/libSceAgc/DcbState/src/Init.cpp L14-23 core/libs/prx/libSceAgc/tests/Commands.cpp L627-632 docs/dev/debt/libSceAgc.md L10

[mazen320]

Re the overlap bot vs #5889: Depends on: none — this is a competing change to the same exports, not a base we build on.

	This PR (#5163)	#5889
sceAgcInit ABI kept as uint32_t	yes	changes to (void*, uint32_t)
Out-of-range / pointer-shaped >=14	return 0, no GetRegisterDefaults	still GetRegisterDefaults + 0x80890001 above 13
Matches MarcZX100 / #4808 probe guidance (pointer-shaped success without table lookup)	yes	no
babyhuey Windows evidence on this PR (PPSA99995 window + PPSA99011 READY)	yes	n/a

Same files because both touch Init.cpp / tests / the sceAgcInit debt bullet. Same exports: sceAgcInit / sceAgcInit_0090. Merging one supersedes the other for those lines — second merger rebases or closes as duplicate.

Prefer this PR for the out-of-range path reviewers already asked for; #5889 can close or retarget if maintainers want the two-arg signature separately (still needs the >=14 → 0 without lookup to unblock the same titles).

[adriadam10]

Checked the diff against the #4808 console measurement (sceAgcInit(&buf, 8) returns 0, writes nothing): values >=14 return 0 with no register-default lookup, sceAgcInit_0090 checks the pointer and delegates, and the debt entry cites the probe and the scalar 2–13 difference. agc_commands builds and passes with the PR (libSceAgc scope, 22/22) and fails without the change; the Windows/AMD title runs in the thread cover the rest. Merging once CI is green.