Natural Language Processing (Almost) from Scratch

3.4.1 Word-Level Log-Likelihood

1. What is the word-level log-likelihood (WLL) criterion?
   Each word is classified independently: a softmax over the network’s tag scores gives \(p(i \mid x, \theta)\), and \(\log p(y \mid x, \theta) = [f_\theta]_y - \text{logadd}_j [f_\theta]_j\), where \(\text{logadd}_i z_i = \log \sum_i e^{z_i}\). It ignores dependencies between neighbouring tags.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.4.1, p. 2506