AI Models Collapse When Trained on Recursively Generated Data

Model collapse in language models

1. How was model collapse tested on language models?
   OPT-125m was fine-tuned on wikitext2, then used to regenerate a dataset of the same size (continuing each 64-token block), on which the next generation was fine-tuned, and so on, either with no original data or keeping a random 10% of the original data each generation.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Model collapse in language models, pp. 757–758
2. What happened to generated text and perplexity over generations?
   Later generations produced more sequences the original model found likely, but also a growing tail of sequences it would never produce: accumulated errors. Text degenerated, e.g. by generation 9 a passage on church architecture had become a list of jackrabbits with differently coloured tails.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Model collapse in language models, p. 758
3. Did keeping some real data or penalizing repetition prevent collapse?
   Keeping 10% of the original data reduced degradation to minor levels but didn’t eliminate it. Penalizing repetition made models produce lower-probability continuations and the next generations perform even worse (perplexity doubled).
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), Model collapse in language models, p. 758