fix(shader): correct v_log_f32 near 1.0 when the driver's log2 strays from a series

[SleglJan]

What

v_log_f32 is lowered to the driver's Log2. Near 1.0 the result tends to zero while the driver's absolute error does not: on an RTX 3080 (615.71.09) v_log_f32 of 1 − 2^-24 is 0xb24b4000 (−1.2e-8) where the hardware returns 0xb3b8aa3c (−8.6e-8), 23,947,836 ULP off, and on 16,384 inputs spaced 2^-15 apart from 0.875 to 1.375 it returns the hardware's bits for 1,685 and is up to 29,121 ULP off. RADV (Mesa 26.2.4) returns the hardware's bits on all of them.

Measured on RDNA2 (gfx1036, Ryzen 7950X3D iGPU) with tools/hw-oracle (Linux, CachyOS kernel 7.2.9, hsa-rocr 7.2.4, clang 23.1.1 as the assembler; IEEE 0, f32 denormals flushed, f16 kept, DX10_CLAMP 1, RNE) on 2,368 rows 1 ± 2^-k (1 + m/64) for k = 1..24 and on the dense sample above: the hardware's log2 is within 1 ULP of the correctly rounded value on every input (equal to it on 75 % of the 2,368 rows and 87 % of the dense sample), with no linear rule like the sine's. An eight-term series in d = x − 1 (exact by Sterbenz's lemma for x in [0.5, 2]), evaluated with fused multiply-adds and then multiplied by d and by 1/ln 2, is within 2 ULP of the hardware and of the correctly rounded value on all 9,280 distinct inputs within 1/8 of 1.0 other than 1.0; six terms reach 9 ULP there, and a band of 1/4 needs eleven terms for the same 2 ULP.

The lowering keeps the driver's Log2 and takes the series only for |x − 1| ≤ 1/8 when the two differ by more than 4 ULP (the bit patterns compared as integers): the hardware and the series are both within 2 ULP of the correctly rounded value, so a driver result more than 4 ULP from the series is the one that is off, and a kept one is within 6 ULP of the hardware. On RADV the native result is never more than 2 ULP from the series, the gate never fires and every bit stays. The trade-off against taking the series unconditionally: that would bound every driver at 2 ULP instead of 6, but move 3,426 of the 8,193 dense inputs in the band off the hardware's bits on RADV by 1 or 2 ULP. The translation's picks for zero, negative, infinite and NaN inputs and the denormal flush are unchanged (the band is [0.875, 1.125]).

v_log_f16 is lowered through the same f32 Log2 and rounded to half, so it takes this path too. Over all 65,536 f16 inputs against the oracle, the change moves one RTX 3080 result: 0x3bed goes from 1 ULP off to the hardware's bits. It moves no RADV result. The series is chosen with OpSelect, not a branch, so it is evaluated for every v_log.

Through the recompiler, v_log_f32 results equal to the hardware bits and the largest ULP distance, before → after on main 57f96fd (the before side is main's driver library from a build that also has two libSceSsl and libSceNet commits; the near-one and dense results are bit-identical to those on the earlier base 0fdda7b):

	RADV (Mesa 26.2.4, the iGPU)	RTX 3080 (615.71.09)
sweep, 32,832 inputs (every exponent with 64 mantissas of both signs, specials)	32,832 → 32,832	24,593, max 23,947,836 → 24,598, max 6, 99.5 % within 1 ULP
dense, 16,384 inputs from 0.875 to 1.375	16,384 → 16,384	1,685, max 29,121 → 5,259, max 9
1 ± 2^-k (1 + m/64), 2,368 rows	2,368 → 2,368	51, max 23,947,836 → 1,039, max 5 inside the band and 8 outside it

The RTX 3080's remaining 9 ULP (0x3f900b00, 1.1253), the 8 ULP of the 1 ± 2^-k rows (0.8516) and the sweep's 6 ULP (0.873) are its own log2 outside the band; inside it the kept driver values reach 6 ULP on 8 of the 8,193 dense inputs. The other seven f32 ops of the transcendental sweep give the same results before and after.

Data (oracle captures, the hosts before and after input by input on 0fdda7b, scripts): https://github.com/SleglJan/AnyPS5/tree/audit/oracle-2026-10-08/oracle-audit/log.

Tested

Linux (CachyOS, kernel 7.2.9, GCC 16.2.1), Release with BUILD_TESTING=ON and ANYPS5_ENABLE_SPIRV_TOOLS=ON, RTX 3080 (NVIDIA 615.71.09) and RADV (Mesa 26.2.4, the iGPU), this branch on main 57f96fd.

• New agc_driver_log_near_one: 64 lanes, f32 and f16 in each (1 ± 2^-k for k = 1..24, 1.0, 0.875, 1.125, 0.9, 1.1, 0.5, 2, 3, 10, 0.1, 1e-3, 1e3, the smallest normal, the largest finite, +0 and −1), pinned on the oracle in the titles' mode and run in FLOAT_MODE 0xc0; zero, infinite and NaN results must match the bits, the others are compared in ULPs with a per-lane tolerance of 6 inside the band (the lowering's bound; 49 lanes including 0.875, 1.125, 0.9 and 1.1), 12 outside it and 2 for the smallest normal and the largest finite (measured maxima after the change: RADV 0 everywhere, NVIDIA 4 inside the band at 0.9375 and 3 outside). Written first: with main 57f96fd's driver library it passes on RADV and fails on the RTX 3080 at lane 4 (v_log_f32 of 1.125 is 0x3e2e00db, expected 0x3e2e00d2, 9 ULP); with the change it passes, three repeats on each GPU.
• agc_driver_vop1_float_unary, agc_driver_float16_misc, agc_driver_float16_result_modifiers, agc_driver_formatted_store and agc_driver_typed_heaps pass on both GPUs, three repeats with agc_driver_log_near_one. Full ctest, 702 tests: on the RTX 3080 697 pass, four video_out_flip_* tests skip (no Vulkan in SDL's offscreen video driver) and agc_driver_dcc_keys_first_write fails (agc_driver_dcc_keys_first_write fails where host imports stay write-watched (NVIDIA, Linux): cause and patch #2767); on RADV 694 pass, seven video-out tests skip and agc_driver crashes with a segmentation fault. It crashes intermittently without this change too: 7 of 8 standalone runs from a build of main 57f96fd with two unrelated libSceSsl and libSceNet commits, against 2 of 6 with this branch.
• tools/check_conventions.py --base upstream/main, tools/check_test_registration.py and git diff --check pass.
• Fork CI at 0b26546 (the final commit differs only in TechnicalDebt.md), https://github.com/SleglJan/AnyPS5/actions/runs/38069705713: Linux with lavapipe 702/702 (34 skipped), agc_driver_log_near_one passing; Windows 710/710 with the AGC GPU tests, agc_driver_log_near_one among them, skipped (no Vulkan device); relinker matrix green. With an earlier in-band tolerance of 2, the lavapipe job failed lane 9 (0.96875) at 4 ULP, which is why the tolerance is the lowering's bound.

Overlap: #2380 (merged) left the transcendental TechnicalDebt entry with the clause "except v_log_f32 near 1.0, where its log2 is up to 24 million ULP off"; this commit replaces it with a pointer to the new entry that follows. No open PR touches EmitFPLog2. #1250 and #1491 change other functions of SpirvAluEmitterMath.cpp; that file merges with both without conflict.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq

[Dextroid17]

This is the most rigorous PR I have read in this repo. The measurements name two independent stacks — the NVIDIA driver version and RADV's Mesa version — plus the hardware oracle, the exact FLOAT_MODE bits, and the instruction counts. The defect is stated in ULP at named inputs. And the alternatives are quantified rather than asserted: six terms give 9 ULP, an eight-term series gives 2, and a 1/4 band would need eleven for the same. That last part is what makes this reviewable rather than tasteful.

The series is the right one, and it can be checked as such. Read out of the code, the coefficients are 1, -1/2, 1/3, -1/4, 1/5, -1/6, 1/7, -1/8, evaluated by Horner with fused multiplies and then multiplied by d and by 1/ln 2 — the Mercator series for log(1+d), which is exactly what log2(1+d) needs. d = x - 1 is exact on [0.5, 2] by Sterbenz's lemma, which is what lets the band be this tight, and 1/8 sits inside that guarantee.

The selection is careful where it matters. Comparing the driver's result and the series as integers, and taking the unsigned minimum of the two subtraction directions, is the standard ULP-distance idiom. It is valid here for both signs because the two values it compares — the driver's log2 and the series — share a sign inside the ±1/8 band, and for same-sign bit patterns subtracting the smaller from the larger gives the count either way. Requiring both "within 1/8" and "more than 4 ULP apart" leaves the driver in charge of everything it already gets right, and lands the correction only where it is needed.

One thing I would want in the entry: the cost is unconditional. Because the choice is a Select, the series is evaluated for every v_log_f32, including inputs far outside the band where the result is then discarded. On a GPU that is usually the better trade against divergent branching, so I take it as deliberate — but it is exactly the kind of decision that gets relitigated later, and one clause would settle it. Relatedly, the debt entry groups v_log for f32 and f16 together while this corrects f32 only; if f16 is deliberately left on the driver's approximation, that is worth saying for the same reason.

Written with AI assistance.

[SleglJan]

@Dextroid17 thanks for checking the series and the selection. Both points are now in the TechnicalDebt entry (55d421f) and the description:

• The cost is unconditional. The series is evaluated for every v_log and chosen with OpSelect. That is seven fused multiply-adds and two multiplies per instruction, whether or not the input is in the band.
• f16 does not stay on the driver's approximation. v_log_f16 is lowered through the same f32 Log2 and rounded to half, so the correction reaches it too. I measured it over all 65,536 f16 inputs against the RDNA2 oracle, before (57f96fd) and after (699ff87), in the same mode as the original sweep:
  • RTX 3080 (615.71.09). One result changes: 0x3bed (0.9907) goes from 1 ULP off onto the hardware's bits. Its exact log2 is 0.006 f16 ULP past a rounding midpoint (45 f32 ULP), and NVIDIA's f32 log2 there was 61 ULP off, enough to cross it. One other input, 0x3489, is outside the band and stays 1 ULP off.
  • RADV (Mesa 26.2.4). Every result matches the hardware, before and after.

AI-assisted: yes (Claude Code)

[adriadam10]

Thanks for adding the f16 and OpSelect notes. The code is the same as 699ff87, so the oracle checks I ran on that head still apply (gfx1035, IEEE 0, f32 denormals flushed, over every f32 input within 1/8 of 1.0, 3,145,728 of them):

• The hardware is within 1 ULP of the correctly rounded log2 on all of them, as you measured.

• The series, fused or split into mul + add, is within 2 ULP of the correctly rounded value everywhere. It is 3 ULP from the hardware at three inputs your 9,280 didn't include:

  input	hardware	series	correctly rounded
  0x3f752e47	0xbd7f2c2a	0xbd7f2c27	0xbd7f2c29
  0x3f7baab0	0xbcc9c32e	0xbcc9c32b	0xbcc9c32d
  0x3f7bc4fb	0xbcc4f0af	0xbcc4f0ac	0xbcc4f0ae

  So a kept driver value is within 7 ULP of the hardware, not 6. A driver that returns the hardware's bits still never triggers the gate (0 of 3,145,728).

• All 128 expected values of agc_driver_log_near_one match the oracle bit for bit.

• The new test passes with the change on RADV and on CI's lavapipe (Mesa 25.2.8). Without the change it fails on lavapipe, so Linux CI covers it.

I added one commit on your branch and rebased it on current main:

• docs(technical-debt): your new entry keeps its f16 and OpSelect sentences, and now gives the exhaustive figures (the series is within 3 ULP of the hardware at those three inputs, so the bound is 7 ULP). It also says the RADV bit-for-bit result covers the 9,280 inputs you measured on gfx1036.

The build and the AGC ctest scope pass after the rebase. The full Build's Linux job has two failures that this PR doesn't cause. agc_driver_image_packed16_load also fails on lavapipe on main without this change. agc_driver_flat_apertures is a 120 s timeout in a test that doesn't use v_log.