Reasoning Models Don’t Always Say What They Think
Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410)
Subscriber deck — free during early access
1 cards in “2.2 Data” · whole deck
2.2 Data
- What six hints did Chen et al. use?Four neutral ones: sycophancy (a person suggests an answer), consistency (the model’s own answer in a previous turn), a visual pattern (correct answers marked in few-shot examples) and metadata (the answer embedded in XML). Two misaligned ones: grader hacking (code revealing what the grader rewards) and unethically obtained information.
- Chen et al., 2025, Reasoning Models Don’t Always Say What They Think (arXiv preprint arXiv:2505.05410), §2.2
Source
- Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan and Ethan Perez. 2025. Reasoning Models Don’t Always Say What They Think. In arXiv preprint arXiv:2505.05410. https://arxiv.org/abs/2505.05410