Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015))
Subscriber deck — free during early access
1 cards in “3.2 Encoder: Bidirectional RNN for Annotating Sequences” · whole deck
3.2 Encoder: Bidirectional RNN for Annotating Sequences
- How are the source annotations \(h_j\) computed in Bahdanau et al.?With a bidirectional RNN: a forward RNN reads \(x_1 \to x_{T_x}\) and a backward RNN reads \(x_{T_x} \to x_1\); \(h_j = [\overrightarrow{h}_j^\top; \overleftarrow{h}_j^\top]^\top\) summarizes both preceding and following words, focused on the words around \(x_j\), since RNNs better represent recent inputs.
- Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015)), §3.2
Source
- Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR 2015). https://arxiv.org/abs/1409.0473