feat(agc): filter converted unorm and float images in the shader

[gjonn]

What

Images whose format has no host equivalent are kept in an R32_UINT image and converted in the shader: 10/11-bit unorm (format 30) and, with #1394, R10_G11_B11 float (format 43). The host sampler cannot filter them, so sampling them threw ("samples or gathers a converted unorm/float image, which needs filtering in the shader and is not implemented"). A title samples R10_G11_B11 float images from several compute shaders.

This filters those images in the shader:

• The materializer decodes the paired sampler into a new EmulatedFilter state on the image:

  • point or bilinear filtering (equal min and mag filters);
  • wrap, mirror, clamp-to-edge, clamp-to-half-border (GL_CLAMP: coordinates clamp to the texture edge, so a bilinear sample takes at most half the border color) or clamp-to-border addressing;
  • the built-in border color (transparent black, opaque black, opaque white);
  • the mip filter.

  The state is part of the specialization and the disk-cache entry, and FormatVersion goes from 7 to 8.

• The emitter fetches the four (or one) texels with OpImageFetch, unpacks them with the existing converted-image path (UnpackImageTexel, including the view swizzle) and blends them.

  • The border color replaces the swizzled texel: AMD hardware does not swizzle border colors, which is what VK_EXT_border_color_swizzle describes.
  • Texel offsets and 2D array layers are honoured.
  • 3D views use the sampler's Z addressing mode and blend the two nearest slices (bilinear) or pick one (point).

• Levels:

  • Single-level images, _lz samples and a sampler without a mip filter use the base level.
  • An explicit LOD (image_sample_l) is clamped to the image's levels. The mip filter then selects the nearest level (point) or blends the two adjacent levels (linear).

• Images filtered this way no longer get a point-filtered copy of their sampler, since the shader never samples through it.

• Unnormalized sampler coordinates are used as texel positions on the base level, without scaling by the image size. Images filtered this way are left out of the unnormalized sampler checks, which only apply to host-sampled views.

• Still refused:

  • mirror-once addressing, a border color table, anisotropic or unequal filters and reduction samplers;
  • implicit or clamped LODs across levels, and a sampler or view LOD bias or clamp on a multi-level image;
  • 1D and cube views, bindless tables and gathers.

Tested

• Windows 11, MinGW-w64 GCC 15.2.0, RTX 5090: agc_driver_image_converted_unorm passes with new cases. image_sample_lz across a 32-texel R10_G11_B11 float image, sampled a quarter texel left of each centre, matches a CPU reference for:

  • point with clamp to edge;
  • bilinear with clamp to edge, wrap and mirror;
  • bilinear with half border and a white border;
  • bilinear with unnormalized coordinates;
  • bilinear on a two-slice 3D image sampled halfway between the slices, which returns the average of both slices.

  Gathers of such an image, and mirror-once addressing, are refused.

• A title whose compute shaders sample R10_G11_B11 float images (bilinear, with wrap, mirror, clamp-to-edge and half-border samplers, including an explicit-LOD sample of a two-level image) now gets past them.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: feat(agc): load and store R10_G11_B11 float images #1394 (this branch is stacked on it; review the commits after a64b856)
• AI-assisted: yes

[adriadam10]

Measured on gfx1035 hardware with the HW oracle (format 43, image_sample_lz/image_sample_l, IEEE=0). Build and ctest pass after a local rebase, and the test fails without the change. 2D wrap/mirror/clamp-to-edge/half-border/border addressing and point sampling match the hardware bit for bit at 1/64-texel steps. These parts don't match:

• ResourceMaterializer.cpp:605: the mip filter is read from S# word2 bits 24-25, which hold Z_FILTER. MIP_FILTER is bits 26-27 (see GuestSamplerResource.cpp:84 on main). On a two-level image with image_sample_l, bits 24-25 change nothing. MIP_FILTER=1 switches to level 1 at LOD 0.5, and MIP_FILTER=2 blends. With this PR, a point or linear mip sampler whose Z_FILTER is 0 samples the base level only.
• ResourceMaterializer.cpp:598 / SpirvImageEmitter.cpp:1329: Z_FILTER is not modelled for 3D views. On the hardware, 0 follows the XY filter, 1 is point and 2 is linear. With bilinear XY and Z_FILTER=1, the hardware picks one slice (G = 0, 0, 10, 10 at w = 0.25..0.625 between slices 0 and 10) where the PR blends them. Point XY with Z_FILTER=2 is the opposite case. Either decode it or throw for 1/2.
• SpirvImageEmitter.cpp:1275: the border colour does go through dst_sel on AMD. With swizzle XYZ1 and a transparent black border, the hardware returns (0, 0, 0, 1) and the PR returns (0, 0, 0, 0). With X=ZERO and an opaque white border, the hardware returns 0 in X. The description's claim that AMD does not swizzle border colours doesn't hold. Apply the view swizzle to the border value, at least the constant selectors.
• SpirvImageEmitter.cpp:1313-1314, 1344: the hardware quantizes the bilinear weights to 1/256, rounding to nearest with ties to even (w = 1.5/256 → 2/256, 2.5/256 → 2/256, 3.5/256 → 4/256). The mip-blend weight is quantized the same way (LOD 0.49 → 125/256). Example: texel coordinate 3.00293 gives 0.49609375 on the hardware and 0.49707031 here. In a 1/1024-texel sweep over one texel, 768 of 1024 rows differ.
• SpirvImageEmitter.cpp:1303-1305: TRUNC_COORD (S# word0 bit 27) is ignored. With it set, the hardware matches floor. With it clear (0), the hardware reads texel k at t = k − 2^-12 and t = k − 2^-20, where floor gives k − 1. Model the clear case, or throw for it.
• Test: every sample sits on a quarter texel, which is exactly on the 1/256 grid, and the tolerance is 1e-5, so the test can't catch the weight issue. Please add off-grid positions, a mip-filter case and a border case with a constant selector, with expected values pinned exactly.
• It also conflicts with main now: FormatVersion is already 9 there, so this needs 10.

[adriadam10]

Thanks for the update. This can't be rebased mechanically any more, so it needs a port rather than a conflict fix.

After #639, main has no ResourceSpecialization and no per-resource emulatedCompare in the IR. The emulated depth comparison is now driven by pipeline specialization constants (PipelineSpecialization::CompareBase in DescriptorBindingBuilder.cpp) and by the image modes enumerated in ResourceMaterializer (info.runtimeImageModes, RuntimeImageMode). Applying your three commits on top of main conflicts in ResourceMaterializer.{hpp,cpp}, DescriptorBindingBuilder.cpp and ShaderDiskCache.{hpp,cpp}, all in that redesign.

What the port needs:

• Add the filter as a runtime image mode for the converted formats (30 and 43) and pass the sampler state (addressing, border, mip filter, Z filter) as specialization words, the way the compare state is passed, instead of ResourceSpecialization::Image::emulatedFilter.
• ShaderDiskCache::FormatVersion is 26 on main, so bump to 27 (or max of main and open PRs + 1).

The points from the previous review still apply, measured on gfx1035 with the HW oracle (format 43, image_sample_lz/image_sample_l, IEEE=0):

• Mip filter is read from S# word2 bits 24-25 (Z_FILTER); MIP_FILTER is bits 26-27 (GuestSamplerResource.cpp:84).
• Z_FILTER is not modelled for 3D views: 0 follows the XY filter, 1 is point, 2 is linear. Decode it or throw for 1/2.
• The border colour goes through dst_sel on AMD (XYZ1 with transparent black returns (0, 0, 0, 1)); apply the view swizzle to the border value, at least the constant selectors.
• Bilinear and mip-blend weights are quantized to 1/256, round to nearest even (1.5/256 -> 2/256, 2.5/256 -> 2/256, 3.5/256 -> 4/256).
• TRUNC_COORD (S# word0 bit 27) is ignored: model the clear case or throw for it.
• The test only samples on quarter texels (already on the 1/256 grid) with 1e-5 tolerance. Add off-grid positions, a mip-filter case and a border case with a constant selector, with exact expected values.

Happy to look again once it is on the new pipeline.

[oneandonlydean]

Closing: the points from 10-08 are still open, and since #639 this needs a port onto main's runtime image modes rather than a rebase. If you decide to continue, please reopen.