new MHA v4 entrypoint ; hosts a spectrum of quantized & sparse attention kernels - #4627

Open

jcaraban wants to merge 69 commits into

mainfrom

Commits on Jun 2, 2026

new Sage-like QKi8-Vf8 fMHA asm
jcaraban
committed
73a2b96
extend bench_sage.py with aiter_i8fp8
jcaraban
committed
73bb91f
bench_sage.py format
jcaraban
committed
3233e18
bench_sage: support transformer-like input distribution
jcaraban
committed
3cd60c9
bench_sage: rotate qk and clip qk for better i8fp8 quality
jcaraban
committed
329b9ad
bench_sage: relax thresholds for quantized kernels
jcaraban
committed
85c835d
bench_sage: refactor accuracy metrics report
jcaraban
committed
95387d2
bench_sage: prune non-working logic and default to transformer-like inputs
jcaraban
committed
25d4f33
enable fmha i8fp8 for gfx942
jcaraban
committed
854d2cc
fmha_post_process() bug on mi300 i8fp8 ASM made padded K rows inflate the denominator
jcaraban
committed
25e68ae
optimize i8fp8.co waves balance for +60T
jcaraban
committed
fa82a64
bench_sage.py excludes fav3_fp8 from --kernel all
jcaraban
committed
8fddda7
Revert accidental gfx942 MI300 fp8.co overwrite
jcaraban
committed
562581a
mxfp4 ASM fmha and needed API and bench changes
jcaraban
committed
f8b1677

Commits on Jun 10, 2026

mxfp4 ASM 2400T: hoist schrau constants, schrau fission, K-scale read ahead, speculative softmax
jcaraban
committed
342400a

Commits on Jun 20, 2026

bench_sage: add extra distributions and --seed
jcaraban
committed
ea8757d
mxfp6 ASM fmha and needed API and bench changes
jcaraban
committed
d3c7d8e

Commits on Jun 21, 2026

bench_sage: drop duplicated --seed
jcaraban
committed
ef5aa5f

Commits on Jun 22, 2026

extend mxfp6 packer with coalesced K arrangement
jcaraban
committed
96b5176
refactor mxfp6 packing & add mxfp4 coalesced load
jcaraban
committed
3d73a74

Commits on Jun 23, 2026

refresh bf16 / fp8 / i8fp8 / mxfp4 binaries
jcaraban
committed
de8caa5
split mxfp6 into f6f8 and f6f6 variants
jcaraban
committed
973e99d
mxfp6 K-scale packer: trailing pad byte for op_sel-shift-free kernel
jcaraban
committed
c5fca8c

Commits on Jun 24, 2026

drop f6f6 mha variant, which didn't work out (slower than f6f8)
jcaraban
committed
fd4df16

Commits on Jun 26, 2026

optimize fp6 quant/pack to be more inductor friendly
jcaraban
committed
a21d8ad

Commits on Jul 2, 2026

fmha bf16 hd128: deploy woven fold + front-load build (md5 2fb06ab7)
jcaraban
committed
d2ca346

Commits on Jul 4, 2026

Commits on Jul 6, 2026

fmha mxfp6: split f6f4 (fp6-QK / per-channel fp4-V) into its own .co slot

Chi-Chu319
and
cursoragent
committed
98d962a

Commits on Jul 7, 2026

Updated packer resolve path for f6f4
Chi-Chu319
committed
f75c384

Commits on Jul 19, 2026

Keep sparse attention LUT metadata int32
jcaraban
committed
d7d7d68

Commits on Jul 21, 2026

Commits on Jul 22, 2026

Update gfx950 FMHA production binaries
jcaraban
committed
5aeb8fe

Commits on Jul 23, 2026

Commits on Jul 24, 2026

refined f4f4 v packer
Chi-Chu319
committed
040df76

Commits on Jul 26, 2026

Refresh gfx950 FMHA production binaries
jcaraban
committed
da2f8c3

Commits on Aug 1, 2026

mxfp6 fmha: make compact C1 packing the default
jcaraban
committed
982d412

Commits on Aug 3, 2026

Regenerate gfx950 Sage FMHA binaries
jcaraban
committed
cba008a

Commits on Aug 4, 2026

Commits on Aug 6, 2026

Merge origin/main into mxfp6_fmha_gfx950
jcaraban
committed
c885826
feat(fmha): add gfx950 f4f4 and f6f4 kernels
jcaraban
committed
e84f7f0
fix(fmha): use unique mixed kernel symbols
jcaraban
committed
8ccca03

Commits on Aug 7, 2026

feat(mha): add explicit MHA v4 kernel entrypoint
jcaraban
committed
a601775
[ruff] violations reported by bot
jcaraban
committed
a693d37
moar [ruff] violations
jcaraban
committed
7519a7c
refactor(mha): isolate v4 quant preprocessing
jcaraban
committed
2f39b8b
drop unrelated changes to .gitignore and codegen.py
jcaraban
committed
195dcff
fix(mha): declare v4 output mutation precisely
jcaraban
committed
644da6f
perf(mha): fuse MXFP6 key quantization
jcaraban
committed
287e291

Commits on Aug 8, 2026

Commits on Aug 9, 2026

docs(mha): record MXFP6 K fusion constraints
jcaraban
committed
b0203c6
perf(mha): add true MXFP4 value path
jcaraban
committed
50c3dfc

Commits on Aug 10, 2026