s1: Simple test-time scaling
Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing)
Subscriber deck — free during early access
1 cards in “2.1 Initial collection of 59K samples” · whole deck
2.1 Initial collection of 59K samples
- How was s1’s initial pool of 59K questions built?From 16 sources chosen for quality, difficulty and diversity (mostly NuminaMATH, plus past AIME problems, Olympiad and standardized-test questions, and two new sets: Stanford statistics PhD qualifying-exam probability questions and hard quant-interview brain-teasers). Reasoning traces and answers came from Gemini 2.0 Flash Thinking, decontaminated against the evaluations with 8-grams.
- Muennighoff et al., 2025, s1 (Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing), §2.1, p. 20276
Source
- Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 20275–20321. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2025.emnlp-main.1025 https://aclanthology.org/2025.emnlp-main.1025/