Let’s Verify Step by Step
Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024))
Subscriber deck — free during early access
1 cards in “5 OOD Generalization” · whole deck
5 OOD Generalization
- Did the PRM’s advantage survive distribution shift?Yes: on 224 recent AP Physics, Calculus, Chemistry and AMC10/12 problems released after pretraining data was collected, the PRM again beat the ORM and majority voting.
- Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024)), §5
Source
- Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.20050