AI Models Collapse When Trained on Recursively Generated Data

What is model collapse?

1. What three sources of error drive model collapse?
   Statistical approximation error, from finite samples (rare events may not be sampled at all), the primary cause; functional expressivity error, from limited model capacity (e.g. fitting two Gaussians with one); and functional approximation error, from the learning procedure itself, such as biases of SGD or the objective.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), What is model collapse?, pp. 755–756
2. Why can more model capacity make model collapse worse rather than better?
   Expressiveness can counteract statistical noise but can equally fit and compound it: an overfitted density model may extrapolate and assign high density to regions outside the training support, which are then sampled with arbitrary frequency in the next generation.
   • Shumailov et al., 2024, AI models collapse when trained on recursively generated data (Nature, vol. 631), What is model collapse?, p. 756