Let’s Verify Step by Step
Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024))
Subscriber deck — free during early access
1 cards in “6.1 Credit Assignment” · whole deck
6.1 Credit Assignment
- Why does process supervision work better, according to Lightman et al.?Credit assignment: with outcome labels, a reward model must work out for itself where a wrong solution went wrong, and on hard problems most solutions are wrong somewhere, so a negative label says little. Process labels say how many steps were right and exactly which one failed.
- Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024)), §6.1
Source
- Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.20050