RNNs and LSTMs

14.4 Stacked and bidirectional RNNs

1. What is a stacked RNN, and why does it help?
   Multiple RNNs where the output sequence of one is the input of the next. Layers induce representations at different levels of abstraction (like early vision detecting edges, later stages shapes), but training cost rises quickly with depth.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 14, §14.4, p. 312
2. What is a bidirectional RNN?
   Two independent RNNs, one reading left to right and one right to left; at each step their hidden states are concatenated, \(h_t = [h^f_t; h^b_t]\), capturing both left and right context.

   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 14, §14.4, p. 313
3. Why are bidirectional RNNs useful for sequence classification?
   A forward RNN’s final state reflects the end of the sentence more than its beginning; combining the final states of the forward and backward passes represents the whole sequence more evenly.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 14, §14.4, p. 314