learn.ifs.name

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948)

Subscriber deck — free during early access

1 cards in “4.1 Distillation v.s. Reinforcement Learning” · whole deck

4.1 Distillation v.s. Reinforcement Learning

  1. For a 32B model, which worked better: distilling from DeepSeek-R1 or large-scale RL?
    Distillation. Over 10K steps of RL on Qwen-32B-Base reached about QwQ-32B-Preview level, while fine-tuning the same base on R1’s 800k samples (no RL) did much better. Strong reasoning patterns found by large models transfer cheaply, though going beyond them may still need bigger bases and RL.
    • DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948), §4.1

Source

  • DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. In arXiv preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948