Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020))
Subscriber deck — free during early access
2 cards in “4.1 Open-domain Question Answering” · whole deck
4.1 Open-domain Question Answering
- How did RAG do on open-domain QA?New state of the art on Natural Questions (44.5 EM), WebQuestions (45.5), CuratedTrec (52.2) and the T5-comparable TriviaQA split (68.0), beating closed-book T5-11B+SSM and extractive DPR, without salient-span-masking pre-training or a re-ranker.
- Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §4.1
- Why can generating answers beat extracting them, according to Lewis et al.?Documents that contain clues but not the verbatim answer can still contribute, and RAG can produce correct answers absent from all retrieved documents: 11.8% accuracy in such cases on NQ, where an extractive model scores 0.
- Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §4.1
Source
- Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). https://papers.nips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html