Skip to content

feat(export): add functional quantized weight export - #2251

Draft
mxinO wants to merge 12 commits into
mainfrom
mxin/general-real-quant-refit
Draft

feat(export): add functional quantized weight export#2251
mxinO wants to merge 12 commits into
mainfrom
mxin/general-real-quant-refit

Conversation

@mxinO

@mxinO mxinO commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a functional per-weight export API for online real-quant refit.

  • ModelOpt resolves the exact weight quantizer and captures current quantizer state without mutating the module.
  • Opaque state supports topology-aligned merge and select operations.
  • ModelOpt emits canonical Hugging Face packed weights, sidecars, and quantization_config.
  • Reusable export specs contain metadata only; mutable amax and scale tensors are recaptured for each export stream.

The API supports non-integer FP8, MXFP8, MXFP4, NVFP4 variants, and mixed per-layer configurations. Ordinary offline export remains on its existing path. AWQ, SmoothQuant, SVDQuant, and integer weight formats are outside this PR.

Usage

spec = get_quantized_weight_export_spec(module, "weight")
state = capture_quantized_weight_export_state(module, "weight")
exported = export_quantized_weight_tensors(weight, state, dtype)
config = build_hf_quantization_config({"model.layers.0.mlp.weight": spec})

Validation

  • 24 focused functional-export tests passed.
  • 32 existing offline-export tests passed across Transformers, Diffusers, streaming/offload, and AWQ fallback coverage.
  • ModelOpt pre-commit hooks passed.
  • Downstream two-step W4A16/NVFP4 QA-GRPO completed two refits with finite loss and KL.

Dependency

Megatron-Bridge integration: NVIDIA-NeMo/Megatron-Bridge#5812

Checklist

  • Added focused regression tests.
  • Preserved ordinary offline-export behavior.
  • No new dependency.
  • Changelog update, if required before review.

mxinO added 8 commits August 15, 2026 07:44
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
…nt-refit

Signed-off-by: Meng Xin <mxin@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 26, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-2251/

Built to branch gh-pages at 2026-08-29 07:47 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.33333% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.91%. Comparing base (022767c) to head (8802ee7).

Files with missing lines Patch % Lines
modelopt/torch/export/quant_utils.py 85.09% 55 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2251      +/-   ##
==========================================
- Coverage   78.95%   78.91%   -0.05%     
==========================================
  Files         524      524              
  Lines       60866    61186     +320     
==========================================
+ Hits        48058    48284     +226     
- Misses      12808    12902      +94     
Flag Coverage Δ
unit 56.01% <85.33%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mxinO added 4 commits August 26, 2026 01:20
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant