Skip to content

Add standalone layer-wise KV-cache AutoQuant with forward KL - #2272

Draft
meenchen wants to merge 12 commits into
mainfrom
agent/kv-cache-autoquant-core
Draft

Add standalone layer-wise KV-cache AutoQuant with forward KL#2272
meenchen wants to merge 12 commits into
mainfrom
agent/kv-cache-autoquant-core

Use calibration-free KV AutoQuant candidates

efc25b3
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project failed Aug 27, 2026 in 0s

76.71% (-2.24%) compared to fbcdc16

View this Pull Request on Codecov

76.71% (-2.24%) compared to fbcdc16

Details

Codecov Report

❌ Patch coverage is 91.27660% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.71%. Comparing base (fbcdc16) to head (efc25b3).
⚠️ Report is 16 commits behind head on main.

Files with missing lines Patch % Lines
modelopt/torch/quantization/kv_cache_auto_quant.py 94.88% 16 Missing ⚠️
modelopt/torch/export/quant_utils.py 77.61% 15 Missing ⚠️
modelopt/torch/export/unified_export_hf.py 64.70% 6 Missing ⚠️
modelopt/torch/export/model_utils.py 71.42% 2 Missing ⚠️
...delopt/torch/export/unified_export_hf_streaming.py 50.00% 1 Missing ⚠️
modelopt/torch/quantization/model_quant.py 97.05% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2272      +/-   ##
==========================================
- Coverage   78.94%   76.71%   -2.24%     
==========================================
  Files         522      524       +2     
  Lines       60550    63901    +3351     
==========================================
+ Hits        47803    49019    +1216     
- Misses      12747    14882    +2135

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
View more details on Codecov