A Neural Probabilistic Language Model

Bengio et al., 2003, A Neural Probabilistic Language Model (Journal of Machine Learning Research)

Subscriber deck — free during early access

1 cards in “4.1 N-Gram Models” · whole deck

4.1 N-Gram Models

Which n-gram baselines did Bengio et al. compare against?
A deleted-interpolation trigram with mixture weights conditioned on context frequency bins (trained by EM on validation data), modified Kneser-Ney back-off n-grams, and class-based n-grams, with order and number of classes chosen on validation data.
- Bengio et al., 2003, A Neural Probabilistic Language Model (Journal of Machine Learning Research), §4.1, pp. 1147–1148

Source

Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin. 2003. A Neural Probabilistic Language Model. In Journal of Machine Learning Research, pp. 1137–1155. https://www.jmlr.org/papers/v3/bengio03a.html