Sequence to Sequence Learning with Neural Networks

Sutskever et al., 2014, Sequence to Sequence Learning with Neural Networks (Advances in Neural Information Processing Systems 27 (NIPS 2014))

Subscriber deck — free during early access

1 cards in “5 Conclusion” · whole deck

5 Conclusion

What general lesson did Sutskever et al. draw from source reversal?
Find a problem encoding with the greatest number of short-term dependencies, since they make learning much simpler. They suspected even a standard RNN could be trained on reversed translation data, though they didn’t verify it.
- Sutskever et al., 2014, Sequence to Sequence Learning with Neural Networks (Advances in Neural Information Processing Systems 27 (NIPS 2014)), §5

Source

Ilya Sutskever, Oriol Vinyals and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27 (NIPS 2014). https://papers.nips.cc/paper/2014/hash/5a18e133cbf9f257297f410bb7eca942-Abstract.html