Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020))
Subscriber deck — free during early access
1 cards in “2.5 Decoding” · whole deck
2.5 Decoding
- How are RAG-Token and RAG-Sequence decoded?RAG-Token is an ordinary autoregressive model with per-step marginalized probabilities, so standard beam search works. RAG-Sequence’s likelihood doesn’t factor per token: run beam search per document, then score each candidate across documents (“thorough decoding”, extra forward passes) or approximate missing scores as zero (“fast decoding”).
- Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §2.5
Source
- Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). https://papers.nips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html