Natural Language Processing (Almost) from Scratch

Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research)

Subscriber deck — free during early access

1 cards in “4.4 Embeddings” · whole deck

4.4 Embeddings

What did the language model embeddings look like?
Nearest neighbours were syntactically and semantically related: France near Austria, Belgium, Germany; XBox near Amiga, PlayStation; reddish near greenish, bluish. Supervised-only embeddings had shown no such structure.
- Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §4.4, pp. 2513–2514

Source

Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. In Journal of Machine Learning Research, pp. 2493–2537. https://www.jmlr.org/papers/v12/collobert11a.html