Neural Networks

Introduction

1. What is a feedforward network, and why is modern neural network work called deep learning?
   A network in which computation proceeds from one layer of units to the next, with no cycles. “Deep” because modern networks often have many layers.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 6, Introduction, p. 151
2. How does the power of neural networks compare to logistic regression?
   They are more powerful: even a network with a single hidden layer can in principle learn any function.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 6, Introduction, p. 151
3. How does using neural networks differ from using logistic regression in practice?
   Logistic regression relies on rich hand-designed feature templates. Neural networks usually take raw tokens as input and learn to induce features themselves (representation learning), so they suit tasks with enough data to learn features automatically.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 6, Introduction, p. 151