Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

2.2 Retriever: DPR

1. How does RAG’s retriever score documents?
   Dense Passage Retrieval, a bi-encoder: \(p_\eta(z \mid x) \propto \exp(d(z)^\top q(x))\), with \(d(z) = \text{BERT}_d(z)\) and \(q(x) = \text{BERT}_q(x)\), both BERT-base. Finding the top-\(k\) documents is maximum inner product search (MIPS), solved approximately in sublinear time.
   • Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §2.2