A reproducible RAG retrieval test shows that retrieval performance drops by 40 points when the number of distractor passages grows to 2.68M. The author compares BM25, embedding retrieval, and parent-document approaches, and analyzes why an answer’s position in the context affects the results. The article argues that hand-written evaluations tend to overestimate search performance.
Source:Towards Data Science · Read Original
Leave a Reply