Large Language Diffusion Models
Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025))
Subscriber deck — free during early access
1 cards in “5 Conclusion and Discussion” · whole deck
5 Conclusion and Discussion
- What limitations of LLaDA do its authors note?Generation length must be set by hand; the comparison with autoregressive models on identical data stopped below \(10^{23}\) FLOPs; there is no KV cache or other system optimization, and sampling algorithms are still basic; and it has not had RL alignment. It is also smaller and trained on less data than leading LLMs.
- Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025)), §5
Source
- Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen and Chongxuan Li. 2025. Large Language Diffusion Models. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025). https://arxiv.org/abs/2502.09992