Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015))
Subscriber deck — free during early access
1 cards in “6.1 Learning to Align” · whole deck
6.1 Learning to Align
- How does Bahdanau attention differ from Graves’s (2013) alignment for handwriting synthesis, and what is its cost?Graves’s Gaussian-mixture weights move monotonically in one direction, too restrictive for translation, which needs (long-distance) reordering. Bahdanau attention computes a weight for every source word for every target word: \(O(T_x T_y)\), fine for 15–40-word sentences but possibly limiting for other tasks.
- Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015)), §6.1
Source
- Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR 2015). https://arxiv.org/abs/1409.0473