Latent Dirichlet Allocation

5.4 Smoothing

1. How does smoothed LDA handle words unseen in training?
   Maximum likelihood gives unseen words zero probability, and Laplace smoothing isn’t justified as MAP in a mixture model. Instead each row \(\beta_i\) gets an exchangeable Dirichlet prior with parameter \(\eta\) and is treated as random, with variational posterior \(\text{Dir}(\lambda_i)\), \(\lambda_{ij} = \eta + \sum_d \sum_n \phi^*_{dni} w^j_{dn}\): a fuller Bayesian treatment.
   • Blei et al., 2003, Latent Dirichlet Allocation (Journal of Machine Learning Research), §5.4, pp. 1006–1007