learn.ifs.name

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Shao et al., 2024, DeepSeekMath (arXiv preprint arXiv:2402.03300)

Subscriber deck — free during early access

1 cards in “5.2.1 Towards to a Unified Paradigm” · whole deck

5.2.1 Towards to a Unified Paradigm

  1. What unified view of training methods do Shao et al. propose?
    SFT, rejection-sampling fine-tuning, DPO, PPO and GRPO all compute a gradient as a coefficient times \(\nabla \log \pi_\theta\) of tokens, differing in three components: the data source (offline or sampled online), the reward function (rule, model, none) and the algorithm turning data and reward into the gradient coefficient.
    • Shao et al., 2024, DeepSeekMath (arXiv preprint arXiv:2402.03300), §5.2.1

Source

  • Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. In arXiv preprint arXiv:2402.03300. https://arxiv.org/abs/2402.03300