Lost in the Middle: How Language Models Use Long Contexts
Liu et al., 2024, Lost in the Middle (Transactions of ACL, vol. 12)
Subscriber deck — free during early access
2 cards in “5 Is More Context Always Better? A Case Study With Open-Domain QA” · whole deck
5 Is More Context Always Better? A Case Study With Open-Domain QA
- What did the open-domain QA case study show about adding retrieved documents?Reader accuracy saturated long before retriever recall did: going from 20 to 50 retrieved documents gained only about 1–1.5 points while greatly increasing context length, latency and cost.
- Liu et al., 2024, Lost in the Middle (Transactions of ACL, vol. 12), §5, p. 166
- What practical remedies for retrieval-augmented generation do Liu et al. suggest?Rerank retrieved documents so relevant ones come near the start of the context, and truncate the ranked list, retrieving fewer documents when appropriate.
- Liu et al., 2024, Lost in the Middle (Transactions of ACL, vol. 12), §5, p. 166
Source
- Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. In Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173. License: CC BY 4.0. Flashcards adapted from this work. doi:10.1162/tacl_a_00638 https://aclanthology.org/2024.tacl-1.9/