Natural Language Processing (Almost) from Scratch

3.3.2 Sentence Approach

1. Why does SRL need the sentence approach rather than a window?
   A word’s role depends on a chosen predicate, which may lie outside the window. The sentence approach looks at the whole sentence, adding to each word features for its relative distance to the predicate and to the word being tagged.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.3.2, pp. 2503–2504
2. How does the sentence approach network turn a variable-length sentence into a fixed-size vector?
   Convolutional (time-delay) layers apply the same linear map to every window, giving local features per position; a max over positions then keeps, for each feature, its strongest value anywhere in the sentence. An average would make little sense, since most words don’t affect a given word’s role.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.3.2, pp. 2503–2504