learn.ifs.name

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Shao et al., 2024, DeepSeekMath (arXiv preprint arXiv:2402.03300)

Subscriber deck — free during early access

1 cards in “4.1.3 Process Supervision RL with GRPO” · whole deck

4.1.3 Process Supervision RL with GRPO

  1. How does GRPO use process supervision?
    A process reward model scores each reasoning step of every sampled output; step rewards are normalized across the group, and each token’s advantage is the sum of the normalized rewards of the steps from that token onward.
    • Shao et al., 2024, DeepSeekMath (arXiv preprint arXiv:2402.03300), §4.1.3

Source

  • Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. In arXiv preprint arXiv:2402.03300. https://arxiv.org/abs/2402.03300