learn.ifs.name

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948)

Subscriber deck — free during early access

1 cards in “2.2.1 Reinforcement Learning Algorithm” · whole deck

2.2.1 Reinforcement Learning Algorithm

  1. Which RL algorithm trains DeepSeek-R1-Zero, and what does it save?
    GRPO: for each question it samples a group of outputs and uses each output’s reward, normalized by the group’s mean and standard deviation, as its advantage, so no critic model as large as the policy is needed.
    • DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948), §2.2.1

Source

  • DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. In arXiv preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948