learn.ifs.name

GloVe: Global Vectors for Word Representation

Pennington et al., 2014, GloVe (Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP))

Subscriber deck — free during early access

1 cards in “4.5 Model Analysis: Vector Length and Context Size” · whole deck

4.5 Model Analysis: Vector Length and Context Size

  1. How do vector size and context window affect GloVe’s analogy accuracy?
    Returns diminish above about 200 dimensions. Small and asymmetric (left-only) windows help syntactic analogies, since syntax depends on immediate context and word order; larger windows capture more semantic, non-local information.
    • Pennington et al., 2014, GloVe (Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §4.5, p. 1540

Source

  • Jeffrey Pennington, Richard Socher and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. doi:10.3115/v1/D14-1162 https://aclanthology.org/D14-1162/