[xiaobochen123]

Motivation

fix moe ep bug, when load fp8 model. Links to related issues link

Test model : neuralmagic/DeepSeek-Coder-V2-Instruct-FP8

Accuracy: 0.932
Invalid: 0.000
Latency: 243.824 s
Output throughput: 1027.530 token/s

cc: @ispobock

Modifications

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.