Attention Is All You Need
Vaswani et al., 2017, Attention Is All You Need (Advances in Neural Information Processing Systems 30 (NIPS 2017))
Subscriber deck — free during early access
1 cards in “7 Conclusion” · whole deck
7 Conclusion
- What future directions did the Transformer paper anticipate?Applying attention-based models to other tasks and modalities (images, audio, video), local restricted attention for large inputs and outputs, and making generation less sequential.
- Vaswani et al., 2017, Attention Is All You Need (Advances in Neural Information Processing Systems 30 (NIPS 2017)), §7
Source
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems 30 (NIPS 2017). https://papers.nips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html