Let’s Verify Step by Step
Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024))
Subscriber deck — free during early access
1 cards in “3 Large-scale Supervision” · whole deck
3 Large-scale Supervision
- How did the large-scale PRM compare with the ORM and majority voting?Searching over GPT-4 generator samples, the PRM beat both at every N, and the gap widened as N grew; the PRM solved 78.2% of a representative subset of MATH test problems. The ORM was only slightly better than majority voting.
- Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024)), §3
Source
- Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.20050