Goodfire Analyzes Sparse Neuron Ablation Results for Qwen3.6-35B-A3B

Written by

in

Goodfire’s research report on Qwen3.6-35B-A3B found that retaining only the 320 strongest routing neurons in each layer can zero out 92.2% of activations, while the loss increases from 1.886 to 1.940, a rise of approximately 2.9%. After the model is restored through 91M SFT tokens and 9M RL tokens, its SuperGPQA score increases from 44.8 to 45.4. However, this is a vendor report based on a single MoE architecture and still requires validation through other models and independent third-party replication.

Source:explainx.ai · Read the original

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *