Goodfire’s research report on Qwen3.6-35B-A3B found that retaining only the 320 strongest routing neurons in each layer can zero out 92.2% of activations, while the loss increases from 1.886 to 1.940, a rise of approximately 2.9%. After the model is restored through 91M SFT tokens and 9M RL tokens, its SuperGPQA score increases from 44.8 to 45.4. However, this is a vendor report based on a single MoE architecture and still requires validation through other models and independent third-party replication.
Source:explainx.ai · Read the original
Leave a Reply