learn.ifs.name

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948)

Subscriber deck — free during early access

1 cards in “2.2.2 Reward Modeling” · whole deck

2.2.2 Reward Modeling

  1. What rewards does DeepSeek-R1-Zero get, and why no neural reward model?
    Rule-based ones only: an accuracy reward (e.g. a boxed math answer checked against the solution, code run against tests) and a format reward for putting the reasoning between <think> tags. Neural outcome or process reward models were avoided because large-scale RL tends to hack them, and retraining them adds cost.
    • DeepSeek-AI, 2025, DeepSeek-R1 (arXiv preprint arXiv:2501.12948), §2.2.2

Source

  • DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. In arXiv preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948