Skip to content

refactor(modelopt): delegate real-quant export to ModelOpt - #5812

Draft
mxinO wants to merge 14 commits into
mainfrom
mxin/general-real-quant-refit
Draft

refactor(modelopt): delegate real-quant export to ModelOpt#5812
mxinO wants to merge 14 commits into
mainfrom
mxin/general-real-quant-refit

Conversation

@mxinO

@mxinO mxinO commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Delegates real-quant packing and deployment metadata to ModelOpt while keeping distributed topology and Hugging Face naming in Megatron-Bridge.

  • Reuses existing TP, PP, EP, and ETP conversion tasks for quantized weights.
  • Handles fused QKV, gate/up, and numbered grouped-expert weights.
  • Stores only topology and format metadata in the reusable plan.
  • Recaptures current amax and scale tensors for each export stream.
  • Uses object collectives only for small metadata and typed collectives for tensor state.
  • Forwards all ModelOpt-emitted packed weights and sidecars without format-specific scale tables.

Unmerged adapter export and quantized mappings requiring dimension permutation are rejected explicitly. Ordinary non-ModelOpt export paths are unchanged.

Ownership

  • ModelOpt: format interpretation, state transforms, packing, sidecars, and quantization_config.
  • Megatron-Bridge: topology conversion, expert identity, and canonical Hugging Face names.

Validation

  • 13 focused tests passed, including distributed state transport and expert gathering.
  • ModelOpt and Megatron-Bridge pre-commit hooks passed.
  • Downstream two-step W4A16/NVFP4 QA-GRPO completed two independent refits with finite loss and KL.

Dependency

Requires NVIDIA/Model-Optimizer#2251.

Checklist

  • Added topology, grouped-weight, expert, and state-refresh tests.
  • Updated ModelOpt quantization documentation.
  • Kept optional ModelOpt imports scoped to ModelOpt paths.

mxinO added 10 commits August 15, 2026 08:39
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
…nt-refit

Signed-off-by: Meng Xin <mxin@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 26, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

mxinO added 4 commits August 26, 2026 01:20
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Signed-off-by: Meng Xin <mxin@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant