LoopFill is a training-free multi-token speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions in a single draft tree, reducing the serial decoding latency of Looped Transformers. Across seven benchmarks, compared with greedy autoregressive decoding, LoopFill achieves an average speedup of 5.85× on Ouro-2.6B and 5.13× on Huginn-3.5B, outperforming the strongest baseline, LoopAhead, by 40% and 25%, respectively.
Source:Hugging Face 社区论文精选 · Read original
Leave a Reply