LIMA: Less Is More for Alignment
Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “6 Multi-Turn Dialogue” · whole deck
6 Multi-Turn Dialogue
- What did LIMA show about multi-turn dialogue?Trained on no dialogues, it still held surprisingly coherent conversations but failed within 3 turns in 6 of 10 chats. Adding just 30 dialogue chains raised excellent responses from 45.2% to 76.1% and cut failures from 15 of 42 turns to 1 of 46, again suggesting the ability comes from pretraining.
- Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §6
Source
- Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis and Omer Levy. 2023. LIMA: Less Is More for Alignment. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/ac662d74829e4407ce1d126477f4a03a-Abstract-Conference.html