Natural Language Processing (Almost) from Scratch

7. Critical Discussion

1. How did Collobert et al. answer the criticism “why abandon painfully acquired feature engineering expertise”?
   No single NLP task covers the goals of NLP, so engineering that doesn’t generalize across tasks is undesirable, though they acknowledge their debt to task-specific research. They chose neural networks for their ability to discover hidden representations with stochastic learning that scales linearly with data.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), 7. Critical Discussion, pp. 2527–2528