Attention Is All You Need
Vaswani et al., 2017, Attention Is All You Need (Advances in Neural Information Processing Systems 30 (NIPS 2017))
Subscriber deck — free during early access
1 cards in “3.2 Attention” · whole deck
3.2 Attention
- How does the Transformer paper define an attention function?A mapping from a query and a set of key–value pairs to an output: a weighted sum of the values, where each value’s weight comes from a compatibility function of the query with the corresponding key.
- Vaswani et al., 2017, Attention Is All You Need (Advances in Neural Information Processing Systems 30 (NIPS 2017)), §3.2
Source
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems 30 (NIPS 2017). https://papers.nips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html