s1: Simple test-time scaling
Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing)
Subscriber deck — free during early access
1 cards in “5.1 Data Quantity, Diversity, and Difficulty” · whole deck
5.1 Data Quantity, Diversity, and Difficulty
- What did s1’s data ablations show?Choosing 1,000 samples by quality alone (random), diversity alone, or difficulty alone (longest traces) was clearly worse than s1K, about 30 points lower on AIME24 on average. Training on all 59K was not clearly better, while costing 394 H100 hours instead of 7.
- Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing), §5.1, pp. 20280, 20282
Source
- Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 20275–20321. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2025.emnlp-main.1025 https://aclanthology.org/2025.emnlp-main.1025/