learn.ifs.name

Let’s Verify Step by Step

Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024))

Subscriber deck — free during early access

1 cards in “6.2 Alignment Impact” · whole deck

6.2 Alignment Impact

  1. What is an “alignment tax”, and what did Lightman et al. find for process supervision?
    The performance cost sometimes paid for safer training methods, which discourages adopting them. Process supervision had a negative alignment tax: it was both more interpretable and better performing, at least in math.
    • Lightman et al., 2024, Let’s Verify Step by Step (The Twelfth International Conference on Learning Representations (ICLR 2024)), §6.2

Source

  • Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.20050