Skip to content

fix(shader): quiet signaling NaN halves in v_pack_b32_f16 in IEEE mode - #2042

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
CouchRiv:fix-shader-pack-b32-f16-snan
Oct 10, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
CouchRiv:fix-shader-pack-b32-f16-snan

Conversation

@CouchRiv

@CouchRiv CouchRiv commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What

Part of #981, a follow-up to #1867. In a shader with IEEE_MODE=1, v_pack_b32_f16 returns a signaling NaN half unchanged:

v_pack_b32_f16 v10, 0x0000, 0x7c01   hardware 7e010000, recompiler 7c010000
v_pack_b32_f16 v10, 0x0000, 0xfd23   hardware ff230000, recompiler fd230000

The hardware quiets each signaling NaN half it packs (after neg and abs) when IEEE=1, and copies the bits when IEEE=0. This is the rule #1867 measured for the unary instructions. The recompiler always copied the bits. vPackB32F16 now passes each NaN half through quietNan16. IEEE=0, and shaders without a known float mode, behave as before.

Halves (src0, src1) IEEE=0 IEEE=1
0x7c01, 0x7dff 7dff7c01 7fff7e01
0x0000, 0xfd23 fd230000 ff230000
0x7c01, 0x7fff (op_sel, high halves) 7fff7c01 7fff7e01
-0x7c01, |0x7dff| 7dfffc01 7ffffe01
quiet NaN, inf, normal, denormal unchanged unchanged

Which title needs it

No known title needs it yet. It extends the rule #1867 measured to one more instruction, and it only changes shaders running with IEEE=1, which titles we know do not use (they use 0xc0, IEEE=0).

Tested

CachyOS Linux, GCC 16.2, RTX 4070 Ti (NVIDIA 615.71) and Radeon 610M (RADV, Mesa 26.2.4). Measured on RDNA2 (gfx1036) with the HW oracle.

  • agc_driver_vop3_pack_ldexp gets a second shader: 4 forms of v_pack_b32_f16 (plain, op_sel, neg/abs) × 32 rows of signaling, quiet and ordinary halves. It runs with no mode, IEEE=0 and IEEE=1 and checks the hardware tables bit for bit. The hardware gave the same results with DX10 clamp on or off and f32 denormals kept or flushed.
  • Without the fix the test fails on both drivers: lane 0 IEEE=1 v_pack_b32_f16 v10, v4, v5 is 0x7dff7c01, expected 0x7fff7e01.
  • A differential run of 424 rows (f16 values with signaling NaNs, high-half garbage) in 4 float modes: 78 rows differed with IEEE=1 before, 0 after, on both drivers.
  • Full Release ctest 498/498 on NVIDIA and on RADV; check_conventions.py and git diff --check pass.
  • Not tested: a title, since none is known to run with IEEE=1.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none (fix(shader): quiet signaling NaNs only in IEEE mode #1867 merged)
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* needs rebase Conflicts with main, rebase needed labels Oct 9, 2026
@CouchRiv
CouchRiv force-pushed the fix-shader-pack-b32-f16-snan branch from 3a3b82c to 77cc1da Compare October 9, 2026 08:19
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 9, 2026
@popcoder565

Copy link
Copy Markdown
Contributor

Native Windows run, since no Windows results were posted. Setup: Windows 10 (19045), RTX 3060 Laptop (NVIDIA 616.92), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release. Head 77cc1da2, merged onto main d70b899 in one build with 33 other AGC/shader PRs that had no Windows run.

No title was run.

🤖 Addressed by Claude Code

With IEEE_MODE=1 the hardware quiets each signaling NaN half that
v_pack_b32_f16 packs, after neg and abs; with IEEE=0 it copies the bits.
The recompiler always copied them. Same rule as boykopovar#1867, measured on
gfx1036 in both modes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@adriadam10

Copy link
Copy Markdown
Collaborator

Rebased onto current main (mechanical conflict in docs/dev/TechnicalDebt.md after #2000; your sentence about v_pack_b32_f16 is kept in the "Signaling NaN inputs" entry). The code and test are unchanged. The earlier red linux/windows jobs were agc_shader_fusion_tests failing to find spirv/unified1/spirv.hpp, which main fixed in 37bc6f2; unrelated to this PR. Full build and ctest on the rebased head: 630/630 pass, including agc_driver_vop3_pack_ldexp. Merging once CI is green.

@adriadam10
adriadam10 force-pushed the fix-shader-pack-b32-f16-snan branch from 77cc1da to e1f81ab Compare October 10, 2026 06:56
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@adriadam10
adriadam10 merged commit ca41295 into boykopovar:main Oct 10, 2026
14 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…f16-snan

fix(shader): quiet signaling NaN halves in v_pack_b32_f16 in IEEE mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants