Lost in the Middle: How Language Models Use Long Contexts
Liu et al., 2024, Lost in the Middle (Transactions of ACL, vol. 12)
Subscriber deck — free during early access
1 cards in “6.3 The Serial-Position Effect” · whole deck
6.3 The Serial-Position Effect
- What psychological phenomenon does the U-shaped curve resemble, and why is that surprising?The serial-position effect: in free recall people remember the first and last items of a list best. It is surprising in Transformers, whose self-attention can in principle retrieve any token equally well.
- Liu et al., 2024, Lost in the Middle (Transactions of ACL, vol. 12), §6.3, p. 167
Source
- Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. In Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173. License: CC BY 4.0. Flashcards adapted from this work. doi:10.1162/tacl_a_00638 https://aclanthology.org/2024.tacl-1.9/