Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

4.5 Additional Results

1. What did RAG’s retrieval ablations show?
   Learning the retriever improved all tasks over freezing it. Replacing it with BM25 hurt everywhere except FEVER, whose entity-centric claims suit word-overlap retrieval; differentiable retrieval mattered most for open-domain QA.
   • Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §4.5
2. How can RAG’s knowledge be updated without retraining?
   By hot-swapping the index: with a 2016 Wikipedia index RAG answered 70% of “Who is {position}?” questions correctly for 2016 world leaders, with the 2018 index 68% for 2018 leaders, while mismatched index and year gave 12% and 4%.
   • Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §4.5