have mismatching datatypes. 0x416 != 0x232

[ThreeD-H]

I use

QUANTSIM_CONFIG = "htp_v68"
default_output_bw=16,
default_param_bw=8,
python -m Examples.torch.quantize \
 --model-id "Qwen/Qwen2.5-0.5B-Instruct" \
 --recipe "pcq_spinquant_adascale" \
 --export-path "./torch_pcq" \
 --adascale-num-batches 128 --adascale-num-iterations 2048

exporting onnx was successful.
then use

qairt-converter --input_network '/torch_pcq/model_cl4096.onnx'  --quantization_overrides '/torch_pcq/model_cl4096.encodings'  --output_path /torch_pcq/model.dlc

INFO_CONVERSION_SUCCESS: Conversion completed successfully
then use

qairt-quantizer --input_dlc /torch_pcq/model.dlc --output_dlc /torch_pcq/model_quantized.dlc --float_fallback

INFO - Quantization completed successfully:
then use

qnn-context-binary-generator --model /opt/qcom/aistack/qairt/2.31.0.250130/lib/x86_64-linux-clang/libQnnModelDlc.so --backend /opt/qcom/aistack/qairt/2.31.0.250130/lib/x86_64-linux-clang/libQnnHtp.so --dlc_path torch_pcq/model_quantized.dlc --output_dir  /torch_pcq --binary_file qwen2_5_4096

encountered an error:

qnn-context-binary-generator pid:163501 267.1ms [ ERROR ] Tensor 221 and 119 have mismatching datatypes. 0x416 != 0x232.
[ ERROR ] Op specific validation failed.
[ ERROR ] <E> validateNativeOps master op validator rms_norm_0:qti.aisw:RmsNorm failed 3110
[ ERROR ] <E> QnnBackend_validateOpConfig failed 3110
[ ERROR ] <E> Failed to validate op rms_norm_0 with error 0xc26
[ ERROR ] Failed to successfully compose graph 266.4ms
[ ERROR ] Graph composer failed, no graphs were created 323.1ms
[ ERROR ] Failed in composeGraphs() 323.2ms 
[ ERROR ] ComposeGraphs Failed with error = 1 Graph Compose failure 

and I tried not to use --float_fallback,but it would cause another error:

[ ERROR ] tcm_migration.cc:1879:ERROR:no properties registered for q::QNN_RmsNorm
[ ERROR ] graph_prepare.cc:217:ERROR:could not create op: q::Gather
[ ERROR ] graph_prepare.cc:1412:ERROR:Op 0x2c50dc00000011 preparation failed with err:-1
[ ERROR ]  <E> "/model/model/embed_tokens/Gather" generated: could not create op
[ ERROR ]  <E> RouterX86 graph prepare failed 12
[ ERROR ]  <E> Failed to finalize graph (id: 1) with err 1002
[ ERROR ]  <E> Graph model was not finalized. Cannot create binary

[bhushan23]

Hi ThreeD-H thank you for sharing details on commands used.
few things stand out to me

1. use of 2.31 QAIRT -> this is very old QAIRT SDK and I would recommend upgrading to newest i.e. 2.46
2. For taking LLMs on target, we recommend the following tutorial https://github.com/qualcomm/ai-hub-apps/tree/main/tutorials/llm_on_genie
   a. Recipe from AIMET refers to Quantization recipe to find best model from accuracy point-of-view for your use case

Error you are referring to is coming from RMSNorm dtype mismatch. Could you please confirm if you see the same issue with 2.46