Evolving Machines has released the low-rank Echelon optimizer, reducing the time per step from 33.9 seconds to 3.23 seconds in a 70B-scale layer and achieving up to a 10.5× optimizer speedup. Its 1.3B end-to-end experiments used 3 random seeds and showed that the loss was within the range of seed noise relative to Muon, while the DeepSeek V4.1 Flash projection indicated an approximately 35% reduction in MoE training step time. The GitHub code, Python package, and fused kernels are still pending release, and some MoE timing results do not include communication overhead.
Source:explainx.ai · Read the original
Leave a Reply