Large Language Diffusion Models
Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025))
Subscriber deck — free during early access
1 cards in “2.4 Inference” · whole deck
2.4 Inference
- How does LLaDA generate a response?It starts from a fully masked response of chosen length and runs a set number of steps from \(t = 1\) to \(0\). At each step it predicts all masked tokens, then remasks a share of them, those predicted with the lowest confidence, so the confident tokens stay. More steps cost more but give better text.
Sampling: predict all masks, keep the confident ones, remask the rest (invented example) - Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025)), §2.4
Source
- Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen and Chongxuan Li. 2025. Large Language Diffusion Models. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025). https://arxiv.org/abs/2502.09992