Training language models to follow instructions with human feedback
Ouyang et al., 2022, Training language models to follow instructions with human feedback (Advances in Neural Information Processing Systems 35 (NeurIPS 2022))
Subscriber deck — free during early access
1 cards in “3.2 Dataset” · whole deck
3.2 Dataset
- Where did InstructGPT’s prompts come from, and how big were the data sets?Mostly prompts submitted to the OpenAI API (over 96% English, deduplicated, PII filtered, with users split between train and test) plus some labeler-written ones: about 13k prompts for SFT, 33k for the reward model, and 31k unlabelled API prompts for PPO.
- Ouyang et al., 2022, Training language models to follow instructions with human feedback (Advances in Neural Information Processing Systems 35 (NeurIPS 2022)), §3.2
Source
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022). https://papers.nips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html