Natural Language Processing (Almost) from Scratch
Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research)
Subscriber deck — free during early access
1 cards in “3.3.1 Window Approach” · whole deck
3.3.1 Window Approach
- Describe the window approach network.To tag a word, take a fixed window of \(d_{win}\) words around it (padding at sentence edges), concatenate their lookup vectors, then apply Linear → HardTanh → Linear, giving one score per tag. HardTanh clips its input to \([-1, 1]\), a cheaper tanh.
Each word is looked up in the shared table; the vectors are concatenated - Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.3.1, pp. 2502–2503
Source
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. In Journal of Machine Learning Research, pp. 2493–2537. https://www.jmlr.org/papers/v12/collobert11a.html