s1: Simple test-time scaling
Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing)
Subscriber deck — free during early access
2 cards in “4.2 Performance” · whole deck
4.2 Performance
- What did budget forcing achieve for s1-32B?Accuracy rose with thinking tokens, and forcing longer thinking extrapolated beyond the model’s unforced performance on AIME24, from 50% to 57%, exceeding o1-preview on competition math. Suppressing the end more than about six times flattened out as the model fell into repetitive loops.
- Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing), §4.2, p. 20280
- How did sequential scaling of s1-32B compare with majority voting on the base model?Majority voting over up to 64 samples from Qwen2.5-32B-Instruct never caught up with s1-32B under budget forcing, supporting the intuition that sequential scaling, where later thinking builds on earlier thinking, is more effective than parallel.
- Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing), §4.2, p. 20280
Source
- Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 20275–20321. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2025.emnlp-main.1025 https://aclanthology.org/2025.emnlp-main.1025/