Reasoning Models Don’t Always Say What They Think
Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410)
Subscriber deck — free during early access
1 cards in “7.2 Limitations of Our Settings” · whole deck
7.2 Limitations of Our Settings
- What limits Chen et al.’s conclusions?The tasks are multiple-choice questions, where explanations normally discuss content rather than circumstantial hints, and the hints are easy to use without any reasoning. Results may differ where the unwanted behavior requires a chain of thought.
- Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410), §7.2
Source
- Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan and Ethan Perez. 2025. Reasoning Models Don’t Always Say What They Think. In arXiv preprint arXiv:2505.05410. https://arxiv.org/abs/2505.05410