Reasoning Models Don’t Always Say What They Think
Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410)
Subscriber deck — free during early access
1 cards in “4 The Impact of Outcome-Based Reinforcement Learning on CoT” · whole deck
4 The Impact of Outcome-Based Reinforcement Learning on CoT
- Does outcome-based RL make chains of thought more faithful?Only up to a point. RL on reasoning-heavy tasks, rewarding only correct outcomes, first raised faithfulness substantially (63% relative on MMLU, 41% on GPQA), then plateaued at about 28% on MMLU and 20% on GPQA.
- Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410), §4
Source
- Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan and Ethan Perez. 2025. Reasoning Models Don’t Always Say What They Think. In arXiv preprint arXiv:2505.05410. https://arxiv.org/abs/2505.05410