Add fused MoE support to SelectiveMixedPrecision heuristics

docs/source/features/quantization.md

@@ -226,6 +226,36 @@ are quantized; expert biases remain in full precision.
 }
 ```
 
+## Selective mixed precision for MoE
+
+`SelectiveMixedPrecision` can plan higher precision for the routed fused
+`experts.down_proj` parameters on the explicitly supported Qwen3 and Qwen3.5 MoE
+families. This is available only to the fixed `high_precision_mlp_down` and
+`high_precision_mlp_down_qkv` heuristics; score-based algorithms do not support
+MoE selection. The always-active `shared_expert.down_proj` is not specially
+promoted and remains at the pass default bit width.
+
+MoE planning uses a double opt-in. Set `moe=true` on
+`SelectiveMixedPrecision`, then also set `moe=true` on the first MoE-capable
+PyTorch quantizer (`Rtn`, `KQuant`, or `Gptq`) that consumes the plan. When routed expert
+overrides are emitted, the plan records
+`mixed_precision_info.requires_moe=true`; this requirement prevents a capable
+consumer from silently skipping those overrides. A pass without a `moe` field,
+such as `AutoClip`, may carry the plan forward, and category-only follow-up
+passes may use `moe=false` after a compatible Olive checkpoint with `moe=true`
+already exists. The metadata `default` map never enables or disables the
+consumer's `moe` setting.
+
+`requires_moe` is pass-emitted metadata, not an additional return value from the
+lower-level `get_high_precision_config` or deprecated `get_k_quant_config`
+helpers; both retain their two-value return contract.
+
+This planning metadata belongs to the Hugging Face/PyTorch pass boundary.
+Plain ONNX quantization does not consume it. This stage covers materializing the
+plan into an Olive Hugging Face/PyTorch checkpoint; export through Mobius or ORT
+GenAI `ModelBuilder` is out of scope and has not been validated. Planner
+metadata alone is not an input contract for those export paths.
+
 ## HQQ
 `HQQ (Half-Quadratic Quantization)` is a fast, calibration-free weight quantization method that enables low-bit quantization of large models without relying on gradient-based optimization. Unlike data-dependent approaches like GPTQ, [HQQ](https://dropbox.github.io/hqq_blog/) uses half-quadratic splitting to minimize weight quantization error efficiently.
 

olive/common/hf/wrapper.py

@@ -170,6 +170,17 @@ class LayerWrapper:
     EXPERTS = {
         "default": "experts",
     }
+    # Direct semantic output/down parameter on explicitly supported fused-experts
+    # implementations. There is intentionally no default: consumers of this mapping need
+    # proof that the architecture stores the output projection directly on ``experts`` as
+    # a K-last 3D tensor, rather than guessing from an attribute name.
+    EXPERT_OUTPUTS = {
+        "qwen3_moe": "down_proj",
+        # Both the composite and native text model types are mapped explicitly because
+        # model-type resolution can preserve either form.
+        "qwen3_5_moe": "down_proj",
+        "qwen3_5_moe_text": "down_proj",
+    }
     #
     # Router attribute names verified against transformers 5.14.1:
     #   * ``gate``   -- qwen2_moe, qwen3_moe, mixtral, deepseek_v3, olmoe (the default)
@@ -325,6 +336,41 @@ def get_experts(self, return_name: bool = True):
         name = f"{self.mlp_name}.{self.EXPERTS.get(self.model_type, self.EXPERTS['default'])}"
         return (module, name) if return_name else module
 
+    def get_expert_output(self, return_name: bool = True):
+        """Return the supported fused experts' direct 3D output/down parameter.
+
+        Unlike the general ``get_experts`` accessor, this semantic accessor is deliberately
+        fail-closed. It only recognizes architectures whose fused output parameter topology
+        has been verified, and it rejects per-expert modules and indirect/non-3D parameters.
+        The returned name is local to the experts owner; callers that need a canonical model
+        name must resolve ``(id(experts), parameter_name)`` through ``iter_quant_targets``.
+        """
+        parameter_name = self.EXPERT_OUTPUTS.get(self.model_type)
+        if parameter_name is None:
+            raise ValueError(
+                "Selective mixed precision does not recognize a fused expert output projection "
+                f"for model_type='{self.model_type}'."
+            )
+
+        experts = self.get_experts(return_name=False)
+        if experts is None:
+            raise ValueError("The layer has no resolved experts module.")
+        if isinstance(experts, nn.ModuleList):
+            raise ValueError(
+                "Selective mixed precision requires a fused experts module with a direct 3D "
+                "output projection; per-expert ModuleList topology is unsupported."
+            )
+
+        parameter = dict(experts.named_parameters(recurse=False)).get(parameter_name)
+        if not isinstance(parameter, nn.Parameter):
+            raise ValueError(
+                f"Experts output projection '{parameter_name}' must be a direct nn.Parameter "
+                "on the fused experts module."
+            )
+        if parameter.dim() != 3:
+            raise ValueError(f"Experts output projection '{parameter_name}' must be 3D, got {parameter.dim()}D.")
+        return parameter if not return_name else (parameter, parameter_name)
+
     def get_router(self, return_name: bool = True):
         """Return the router sub-module of this layer (or ``None`` if not MoE).