Distributed Representations of Words and Phrases and their Compositionality
Mikolov et al., 2013, Distributed Representations of Words and Phrases and their Compositionality (Advances in Neural Information Processing Systems 26 (NIPS 2013))
Subscriber deck — free during early access
1 cards in “4.1 Phrase Skip-Gram Results” · whole deck
4.1 Phrase Skip-Gram Results
- What were the phrase analogy results?On 1B words: NEG-15 beat NEG-5, and hierarchical softmax, worst without subsampling, became best with it (47%). With 33B words, hierarchical softmax, 1000 dimensions and whole-sentence context, accuracy reached 72%; 6B words gave 66%, so data size is crucial.
- Mikolov et al., 2013, Distributed Representations of Words and Phrases and their Compositionality (Advances in Neural Information Processing Systems 26 (NIPS 2013)), §4.1
Source
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26 (NIPS 2013). https://papers.nips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html