How LSA Eases the Attention Search Bottleneck in Long-Context AI

Written by

in

Researchers at Prefrontal AI and ETH Zurich propose latent sparse attention (LSA), which clusters keys and values and uses hierarchical search to reduce indexing overhead during the long-context prefill stage. On a 10 million-token task, D-LSA speeds up the prefill of the attention layers in GLM and Qwen by 22.7x and 72.4x respectively, though retrieval accuracy varies across models.

Source:TechTalks · Read the original

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *