Large Language Diffusion Models

1 Introduction

1. What assumption does LLaDA challenge?
   That the core abilities of LLMs (scaling, in-context learning, instruction following) depend on autoregressive next-token prediction. Nie et al. argue they come from the generative modeling principle, maximum likelihood with a large Transformer and data, rather than from the left-to-right factorization.
   • Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025)), §1
2. Which weakness of autoregressive models motivates a non-left-to-right alternative in LLaDA?
   The reversal curse: a model trained left to right struggles to produce what precedes a given text (e.g. the line before a line of a poem), even when it knows the forward direction.
   • Nie et al., 2025, Large Language Diffusion Models (Advances in Neural Information Processing Systems 38 (NeurIPS 2025)), §1