fix(quant): round scaled int8 casts to nearest

[skyguan92]

Summary

• round the shared FP32-to-INT8 scaled casts with __builtin_rintf before the integer conversion instead of truncating toward zero;
• cover both the x2 primitive and the x4 primitive;
• add a deterministic rmsnorm_quant group-64 regression whose RMSNorm factor cancels from the expected value/max ratios.

This aligns the shared HIP helper with torch.round quantization and with AITER's existing OPUS RMSNorm INT8 path, which already uses __builtin_rintf.

Regression

On native gfx1100, the unmodified current-main helper matched truncation on every distinguishing fixture value:

build	values matching RNE	values matching truncation
upstream b3c13c9	0 / 512	512 / 512
this change	512 / 512	0 / 512

The added test was compiled and run from scratch against current AITER main b3c13c932207bef03aa3a8123bf34acad02f40d3:

• test-only baseline: exits 1 at test_i8_group_quant_rounds_to_nearest;
• patched build: exits 0;
• focused existing matrix passes for BF16/INT8 at M=1, N=1024: per-token RMSNorm, add+RMSNorm per-token, and add+RMSNorm group-128.

The fork commit is based on the fork's older upstream ancestor because the available OAuth credential cannot update workflow files. Cherry-picking PR head 35b4d832ac709302719f39589ea8e1bc3efaf19b onto current main produces replay b9f2417ac6d216066825711a48dad777ce5148ae, whose full tree SHA 6428cf8cc322334947af46fa6237d50a5972f5ef is identical to the directly tested current-main commit bdbd025919b3febe36c76b8b572ff7c9285f9f11.

W7900D / DeepSeek V4 evidence

Environment: native gfx1100 with no HSA_OVERRIDE_GFX_VERSION, PyTorch 2.9.1+gitff65f5b, HIP 7.2.53211-e1a6bc5663. PyTorch reports 48 processor units for the selected device.

The layer-local screen used official ModelScope deepseek-ai/DeepSeek-V4-Flash-0731 layer-0 weights, calibrated group-64 INT8 weights, M={1,8,16}, 3 hot samples x 20 repetitions, 20 synchronized 512-MiB shader-read cold samples, and the same W8A8 GEMM on both sides.

projection	local shape (N,K)	quality	max relative L2	fused/current hot p50 geomean	fused/current cold p50 geomean
fused wqkv_a	(1536,4096)	3 / 3	0.009043	1.313x	1.249x
TP8 rank-0 wq_b	(4096,1024)	3 / 3	0.009312	1.289x	1.259x
TP8 rank-0 shared gate/up	(512,4096)	3 / 3	0.009838	1.310x	1.241x

All 9 rows meet cosine >= 0.9995 and relative L2 <= 0.01 and are byte-exact across repeats. With the same wq_b seed, the fused boundary improved from relative L2 0.0129448 before the fix to 0.00992086 after it.

This is not a claim that INT8 beats BF16 at every small projection: wq_b M1/M8 hot results are 0.968x/0.977x versus BF16, and the shared gate/up cold geomean is 0.833x versus BF16. The contribution fixes the bottom-level INT8 contract and makes the existing fusion usable; schedule and model integration work remain separate.

The DeepSeek-shaped performance run used an e261b8f-based AITER source tree. The involved RMSNorm wrapper, HIP kernel, OPUS reference, and pre-patch shared helper blobs are byte-identical to current main b3c13c9; the regression and focused upstream tests above were run directly from b3c13c9.

gfx1100 ISA / resources

For the exact BF16-to-INT8, block-128, no-residual fused kernel selected by the N=1024 test:

• code size: 0xde4 -> 0xe0c bytes (+40 bytes);
• emitted conversion: eight v_rndne_f32 instructions before the integer conversions;
• SGPR: 37 -> 37; VGPR: 22 -> 22;
• SGPR/VGPR spills: 0 -> 0; private segment: 0 -> 0;
• wavefront size: 32 -> 32.

Checks

• Black 26.3.0 check: pass
• Ruff 0.15.7 check: pass
• clang-format 18.1.8 check on the changed C++ hunk: pass
• git diff --check: pass
• DCO sign-off: present

Scope is the shared INT8 conversion contract and its fused RMSNorm regression. It does not change dispatch policy, quantization scale layout, or serving integration.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4637 --add-label <label>