Natural Language Processing (Almost) from Scratch

3.2 Transforming Words into Feature Vectors

1. What is the lookup table layer?
   Words are indices into a dictionary \(\mathcal{D}\); the layer maps word \(w\) to the \(w\)th column of a trainable matrix \(W \in \mathbb{R}^{d_{wrd} \times |\mathcal{D}|}\). Other discrete features (e.g. capitalization) get their own lookup tables, and their vectors are concatenated.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.2, pp. 2501–2502