Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

4.1. Reasoning Paths Already Present in Base Models

1. What evidence shows that RLVR models’ reasoning paths already exist in the base model?
   The problems the RL model solves are nearly a subset of those the base model solves at large \(k\); and the RL model’s responses have low perplexity under the base model, i.e. they are outputs the base model was already likely to produce. RL shifts probability toward rewarded paths rather than creating new ones.
   • Yue et al., 2025, Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (arXiv preprint arXiv:2504.13837), 4.1. Reasoning Paths Already Present in Base Models