Latent Dirichlet Allocation

6 Example

1. What did the 100-topic LDA model on TREC AP news illustrate?
   Topics such as Arts, Budgets, Children and Education emerged as word distributions. For a held-out article about a foundation’s grants to arts and education institutions, four topics had \(\gamma_i - \alpha_i \ge 1\), and per-word \(\phi_n\) showed which topic generated each word.
   • Blei et al., 2003, Latent Dirichlet Allocation (Journal of Machine Learning Research), §6, pp. 1007–1008
2. What limitation of LDA did the AP example reveal?
   The bag-of-words assumption lets words that belong together, like “William Randolph Hearst Foundation”, be allocated to different topics; fixing this requires relaxing exchangeability, e.g. partial exchangeability or Markovian word sequences.
   • Blei et al., 2003, Latent Dirichlet Allocation (Journal of Machine Learning Research), §6, pp. 1007–1008