AI Models Collapse When Trained on Recursively Generated Data

Introduction

1. What question does the model collapse paper ask?
   What happens to future models (“GPT-{n}”) once much of the web text they are trained on was produced by earlier models: models training on their predecessors’ output, generation after generation.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Introduction, p. 755
2. What is model collapse?
   A degenerative process in which data generated by a model pollute the training set of the next generation, which then misperceives reality; models gradually forget the true data distribution, even though the distribution itself doesn’t change.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Introduction, p. 755
3. What are early and late model collapse?
   Early: the model loses information about the tails of the distribution (rare events). Late: it converges to a distribution bearing little resemblance to the original, often with greatly reduced variance, approaching a point estimate.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Introduction, p. 755