Latent Dirichlet Allocation

4.2 Mixture of unigrams

1. How does LDA compare with a mixture of unigrams in assumptions and parameters?
   A mixture of unigrams draws one topic \(z\) per document and all words from \(p(w \mid z)\): each document exhibits exactly one topic, often too limiting. LDA lets documents mix topics in different proportions at the cost of one extra parameter: \(k\) for \(p(\theta \mid \alpha)\) vs \(k - 1\) for \(p(z)\).
   • Blei et al., 2003, Latent Dirichlet Allocation (Journal of Machine Learning Research), §4.2, p. 1000