Skip to content

Add NVFP4/FP8 Q/K/P/V + 2:4 attention quantization for MLA (vLLM TRITON_MLA) - #2244

Draft
kaix-nv wants to merge 5 commits into
mainfrom
kaix/mla_opt
Draft

Add NVFP4/FP8 Q/K/P/V + 2:4 attention quantization for MLA (vLLM TRITON_MLA)#2244
kaix-nv wants to merge 5 commits into
mainfrom
kaix/mla_opt

Add CHANGELOG entry for MLA NVFP4/FP8 + 2:4 attention

f0683c8
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded Aug 25, 2026 in 1s

77.76% (-1.20%) compared to 94915a1

View this Pull Request on Codecov

77.76% (-1.20%) compared to 94915a1

Details

Codecov Report

❌ Patch coverage is 0% with 682 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.76%. Comparing base (94915a1) to head (f0683c8).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...ch/kernels/quantization/attention/mla/reference.py 0.00% 187 Missing ⚠️
.../kernels/quantization/attention/mla/mla_prefill.py 0.00% 154 Missing ⚠️
...h/kernels/quantization/attention/mla/mla_decode.py 0.00% 148 Missing ⚠️
...ch/sparsity/attention_sparsity/plugins/vllm_mla.py 0.00% 89 Missing ⚠️
...parsity/attention_sparsity/plugins/vllm_runtime.py 0.00% 59 Missing ⚠️
modelopt/torch/quantization/plugins/vllm.py 0.00% 36 Missing ⚠️
...t/torch/kernels/quantization/attention/bmm2_qdq.py 0.00% 6 Missing ⚠️
...rch/kernels/quantization/attention/mla/__init__.py 0.00% 3 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2244      +/-   ##
==========================================
- Coverage   78.95%   77.76%   -1.20%     
==========================================
  Files         522      528       +6     
  Lines       60550    62468    +1918     
==========================================
+ Hits        47810    48576     +766     
- Misses      12740    13892    +1152

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
View more details on Codecov