Latent Dirichlet Allocation

5.3 Parameter estimation

1. How are LDA’s parameters \(\alpha\) and \(\beta\) estimated?
   Empirical Bayes by variational EM, maximizing the variational lower bound on \(\sum_d \log p(\mathbf{w}_d \mid \alpha, \beta)\). E-step: fit \(\gamma_d^*\), \(\phi_d^*\) for every document. M-step: \(\beta_{ij} \propto \sum_d \sum_n \phi^*_{dni} w^j_{dn}\), and \(\alpha\) by Newton–Raphson with a Hessian invertible in linear time.
   • Blei et al., 2003, Latent Dirichlet Allocation (Journal of Machine Learning Research), §5.3, pp. 1005–1006