learn.ifs.name

Let’s Verify Step by Step

Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024))

Subscriber deck — free during early access

1 cards in “2.1 Scope” · whole deck

2.1 Scope

  1. How do Lightman et al. evaluate a reward model?
    By best-of-N search: a fixed generator samples N solutions per problem, the reward model picks its top-ranked one, and the fraction of picked solutions with a correct final answer is reported. The generator itself is not trained with RL.
    • Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024)), §2.1

Source

  • Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.20050