Natural Language Processing (Almost) from Scratch

3.5 Supervised Benchmark Results

1. How well did the purely supervised networks do?
   Behind the benchmarks: with SLL, POS 96.37 (vs 97.24), chunking 90.33 (94.29), NER 81.47 (89.31), SRL 70.99 (77.92). SLL beat WLL, mostly on chunking, NER and SRL.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.5, pp. 2508–2510
2. What was wrong with the word embeddings learned by supervised training alone?
   Nearest neighbours in the embedding space weren’t semantically related. Most parameters are in the 100,000 × 50 lookup table, and most words occur too rarely in labeled data to be trained well.
   • Collobert et al., 2011, Natural Language Processing (Almost) from Scratch (Journal of Machine Learning Research), §3.5, pp. 2508–2510