Neural Machine Translation by Jointly Learning to Align and Translate

2.1 RNN Encoder–Decoder

1. How does the basic RNN encoder–decoder define the translation probability?
   The encoder RNN reads \(x_1, \ldots, x_{T_x}\) into hidden states \(h_t = f(x_t, h_{t-1})\) and a vector \(c = q(\{h_1, \ldots, h_{T_x}\})\) (e.g. \(c = h_{T_x}\)); the decoder factorizes \(p(\mathbf{y}) = \prod_t p(y_t \mid \{y_1, \ldots, y_{t-1}\}, c)\), with \(p(y_t \mid \ldots) = g(y_{t-1}, s_t, c)\).
   • Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015)), §2.1