Training language models to follow instructions with human feedback
Ouyang et al., 2022, Training language models to follow instructions with human feedback (Advances in Neural Information Processing Systems 35 (NeurIPS 2022))
Subscriber deck — free during early access
1 cards in “5.2 Limitations” · whole deck
5.2 Limitations
- What is perhaps the greatest limitation of InstructGPT, according to its authors?In most cases it follows the user’s instruction even when that could cause real-world harm: asked to be maximally biased, InstructGPT produced more toxic outputs than GPT-3 of the same size. It is neither fully aligned nor fully safe.
- Ouyang et al., 2022, Training language models to follow instructions with human feedback (Advances in Neural Information Processing Systems 35 (NeurIPS 2022)), §5.2
Source
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022). https://papers.nips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html