Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015))
Subscriber deck — free during early access
2 cards in “4 Experiment Settings” · whole deck
4 Experiment Settings
- What data and setup did Bahdanau et al. use?WMT’14 English–French parallel corpora reduced to 348M words by data selection, no monolingual data; news-test-2014 (3003 sentences) as the test set; a 30,000-word shortlist per language with other words mapped to [UNK]; no lowercasing or stemming.
- Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015)), §4
- Which models did Bahdanau et al. compare?RNNencdec (Cho et al.’s encoder–decoder, 1000 gated hidden units each side) and RNNsearch (bidirectional encoder with 1000 units per direction, 1000-unit decoder, attention), each trained on sentences of up to 30 and up to 50 words; minibatch SGD with Adadelta, about 5 days each, decoding with beam search.
- Bahdanau et al., 2015, Neural Machine Translation by Jointly Learning to Align and Translate (3rd International Conference on Learning Representations (ICLR 2015)), §4
Source
- Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR 2015). https://arxiv.org/abs/1409.0473