LIMA: Less Is More for Alignment
Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
2 cards in “1 Introduction” · whole deck
1 Introduction
- What question does LIMA investigate?How important the two stages of training a language model are relative to each other: unsupervised pretraining versus large-scale instruction tuning and reinforcement learning from human feedback, by aligning a strong pretrained model with only 1,000 examples.
- Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §1
- What is LIMA?LLaMA 65B fine-tuned with the ordinary supervised loss on just 1,000 carefully curated prompts and responses, with no reinforcement learning or preference modelling.
- Zhou et al., 2023, LIMA (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §1
Source
- Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis and Omer Levy. 2023. LIMA: Less Is More for Alignment. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/ac662d74829e4407ce1d126477f4a03a-Abstract-Conference.html