LIMA: Less Is More for Alignment
Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “7 Discussion” · whole deck
7 Discussion
- What limitations of LIMA’s approach do Zhou et al. note?Writing high-quality, diverse examples takes considerable mental effort and is hard to scale; and LIMA is less robust than product models: an unlucky sample or adversarial prompt can produce a weak response.
- Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §7
Source
- Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis and Omer Levy. 2023. LIMA: Less Is More for Alignment. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/ac662d74829e4407ce1d126477f4a03a-Abstract-Conference.html