[GLM-5.2-MXFP4] Retune fMoE kernels with FLAT support

[nholmber]

Motivation

Update the tuned fMoE CSV for GLM-5.2 MXFP4 to improve E2E decode performance on MI355X.

E2E results (SGLang, MI355X, amd/GLM-5.2-MXFP4, FSE=1):

• TP4 (inter_dim=512): TPOT -3.6%** vs base, -4.5% vs upstream CSV
• TP8 (inter_dim=256): TPOT -3.6% vs base, -1.2% vs upstream CSV

Technical Details

Retune FlyDSL kernels for inter_dim=256 (TP8) and inter_dim=512 (TP4). Add FLAT ASM 1-stage kernel entries at low token counts (tok=2-8) and FSE=0 (e=256/k=8) shape coverage.

Test Plan

E2E benchmarks on MI355X using SGLang

• Base image: lmsysorg/sglang-rocm:v0.5.15.post1-rocm720-mi35x-20260714 with AITER updated to 5f40e3b29
• GPU: MI355X (gfx950), TP4 and TP8
• Model: amd/GLM-5.2-MXFP4, --enforce-shared-experts-fusion (FSE=1, e=257/k=9)
• Sweep: ISL=1k/8k/60k, OSL=1k/600, concurrencies 4-256, request-rate=inf

Three configs compared:

• Base: Docker image as is with old AITER 9127c94
• Upstream: new AITER 5f40e3b with the current glm5_fp4_tuned_fmoe.csv
• This PR: new AITER 5f40e3b with the updated CSV

Test Result

TP4 (inter_dim=512) — TPOT (ms)

ISL	OSL	C	Base	Upstream	This PR	vs Base	vs Upstream
1024	1024	4	14.67	15.44	13.66	-6.9%	-11.5%
1024	1024	8	16.31	17.11	16.38	+0.4%	-4.3%
1024	1024	16	19.92	19.77	19.63	-1.4%	-0.7%
1024	1024	32	24.73	24.29	24.10	-2.5%	-0.8%
1024	1024	64	32.48	33.04	32.22	-0.8%	-2.5%
1024	1024	128	43.83	44.61	43.27	-1.3%	-3.0%
1024	1024	256	63.49	63.99	61.58	-3.0%	-3.8%
8192	1024	4	15.50	16.04	14.36	-7.3%	-10.5%
8192	1024	8	18.42	19.18	18.01	-2.2%	-6.1%
8192	1024	16	23.77	23.61	23.34	-1.8%	-1.1%
8192	1024	32	31.80	31.33	30.79	-3.2%	-1.7%
8192	1024	64	47.16	47.88	46.17	-2.1%	-3.6%
8192	1024	128	72.06	72.33	69.13	-4.1%	-4.4%
8192	1024	256	122.65	124.30	118.35	-3.5%	-4.8%
60000	600	4	28.71	29.25	26.87	-6.4%	-8.1%
60000	600	8	48.99	49.53	47.72	-2.6%	-3.7%
60000	600	16	85.95	85.22	82.02	-4.6%	-3.8%
60000	600	32	154.24	154.17	146.46	-5.0%	-5.0%
60000	600	64	214.37	214.67	203.04	-5.3%	-5.4%
60000	600	128	226.54	222.00	210.76	-7.0%	-5.1%

GEOMEAN (n=20)	vs Base	vs Upstream
This PR	-3.6%	-4.5%

TP8 (inter_dim=256) — TPOT (ms)

ISL	OSL	C	Base	Upstream	This PR	vs Base	vs Upstream
1024	1024	4	14.00	13.75	13.37	-4.5%	-2.8%
1024	1024	8	14.94	14.65	13.92	-6.8%	-5.0%
1024	1024	16	17.19	17.52	17.45	+1.5%	-0.4%
1024	1024	32	20.82	20.30	20.23	-2.9%	-0.3%
1024	1024	64	27.62	27.75	27.00	-2.2%	-2.7%
1024	1024	128	37.07	36.64	36.71	-1.0%	+0.2%
1024	1024	256	54.79	53.13	54.08	-1.3%	+1.8%
8192	1024	4	14.87	14.57	13.95	-6.2%	-4.3%
8192	1024	8	17.09	16.72	15.97	-6.6%	-4.5%
8192	1024	16	21.06	21.11	21.13	+0.3%	+0.1%
8192	1024	32	28.40	27.37	27.25	-4.0%	-0.4%
8192	1024	64	43.32	42.39	41.88	-3.3%	-1.2%
8192	1024	128	67.47	64.99	64.80	-4.0%	-0.3%
8192	1024	256	117.38	113.98	115.12	-1.9%	+1.0%
60000	600	4	29.05	28.13	27.45	-5.5%	-2.4%
60000	600	8	49.86	47.77	47.23	-5.3%	-1.1%
60000	600	16	87.94	84.61	84.21	-4.2%	-0.5%
60000	600	32	158.72	152.28	153.03	-3.6%	+0.5%
60000	600	64	296.67	283.55	283.26	-4.5%	-0.1%
60000	600	128	310.60	298.83	295.94	-4.7%	-1.0%

GEOMEAN (n=20)	vs Base	vs Upstream
This PR	-3.6%	-1.2%

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4344 --add-label <label>

[nholmber]

Turning to draft PR until conflict resolved

[nholmber]

Superseded by #4629