learn.ifs.name

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948)

Subscriber deck — free during early access

1 cards in “2.3.1 Cold Start” · whole deck

2.3.1 Cold Start

  1. What is the “cold start” in DeepSeek-R1, and what does it give?
    Before RL, DeepSeek-V3-Base is fine-tuned on thousands of long chain-of-thought examples in a readable format (reasoning followed by a summary), collected by prompting and from cleaned-up R1-Zero outputs. This avoids the unstable early phase of RL from the base model and improved both readability and final performance.
    DeepSeek-V3-Basecold-start SFT (thousands)reasoning RL (GRPO)rejection samplingSFT of V3-Base, 2 epochsRL for all scenariosDeepSeek-R1pure RLR1-Zerono SFT at allits 800k samplesalso distill smalldense models
    The two training pipelines of the DeepSeek-R1 report
    • DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948), §2.3.1

Source

  • DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. In arXiv preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948