A Neural Probabilistic Language Model
Bengio et al., 2003, A Neural Probabilistic Language Model (Journal of Machine Learning Research)
Subscriber deck — free during early access
1 cards in “6 Conclusion” · whole deck
6 Conclusion
- How does the neural LM “fight the curse of dimensionality with its own weapons”?Each training sentence informs the model about a combinatorial number of other sentences through learned distributed representations. Compact, smooth representations replace tables of conditional probabilities, and computation and memory scale linearly, not exponentially, with the number of conditioning variables.
- Bengio et al., 2003, A Neural Probabilistic Language Model (Journal of Machine Learning Research), §6, pp. 1152–1153
Source
- Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin. 2003. A Neural Probabilistic Language Model. In Journal of Machine Learning Research, pp. 1137–1155. https://www.jmlr.org/papers/v3/bengio03a.html