DeepSeek-V3 Technical Report

DeepSeek-AI, 2024, DeepSeek-V3 Technical Report (arXiv preprint arXiv:2412.19437)

Subscriber deck — free during early access

1 cards in “5.2 Reinforcement Learning” · whole deck

5.2 Reinforcement Learning

What rewards did DeepSeek-V3 use in RL, and which algorithm?
Rule-based rewards wherever answers can be checked (boxed math answers, code test cases), being harder to game; a model-based reward model for free-form answers and open-ended tasks. The policy was optimized with GRPO, which needs no critic model.
- DeepSeek-AI, 2024, DeepSeek-V3 Technical Report (arXiv preprint arXiv:2412.19437), §5.2

Source

DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. In arXiv preprint arXiv:2412.19437. https://arxiv.org/abs/2412.19437