learn.ifs.name

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948)

Subscriber deck — free during early access

1 cards in “2.3.3 Rejection Sampling and Supervised Fine-Tuning” · whole deck

2.3.3 Rejection Sampling and Supervised Fine-Tuning

  1. How was the SFT data for DeepSeek-R1’s second fine-tuning built?
    From the converged reasoning-RL checkpoint, several responses were sampled per prompt and only correct, readable ones kept (about 600k reasoning samples, some judged by DeepSeek-V3); about 200k non-reasoning samples (writing, factual QA, translation) came from the DeepSeek-V3 pipeline. DeepSeek-V3-Base was then fine-tuned on all 800k.
    • DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948), §2.3.3

Source

  • DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. In arXiv preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948