Distributed Representations of Words and Phrases and their Compositionality

6 Comparison to Published Word Representations

1. How did the big Skip-gram phrase model compare with earlier published word vectors?
   Its nearest neighbours for rare words (e.g. ninjutsu → ninja, martial arts; capitulate → capitulation, capitulated) were far better than those of Collobert & Weston, Turian et al. and Mnih & Hinton, partly because it was trained on about 30B words, two to three orders of magnitude more, in a fraction of the time.
   • Mikolov et al., 2013, Distributed Representations of Words and Phrases and their Compositionality (Advances in Neural Information Processing Systems 26 (NIPS 2013)), §6