Are Emergent Abilities of Large Language Models a Mirage?
Schaeffer et al., 2023, Are Emergent Abilities of Large Language Models a Mirage? (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “7 Related Work” · whole deck
7 Related Work
- How does Schaeffer et al.’s explanation differ from broken power laws and the “quanta” hypothesis?Broken power laws treat emergence as real, a break in the scaling curve; Schaeffer et al. show it can arise under a single smooth power law. The quanta hypothesis says tasks need sets of discrete skills that larger models are likelier to have all of; both could be true for some abilities.
- Schaeffer et al., 2023, Are Emergent Abilities of Large Language Models a Mirage? (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §7
Source
- Rylan Schaeffer, Brando Miranda and Sanmi Koyejo. 2023. Are Emergent Abilities of Large Language Models a Mirage?. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html