Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

2.1 Models

1. Define RAG-Sequence.
   The same retrieved document generates the whole output: \(p(y \mid x) \approx \sum_{z \in \text{top-}k} p_\eta(z \mid x) \prod_{i=1}^{N} p_\theta(y_i \mid x, z, y_{1:i-1})\), marginalizing over the top-\(k\) documents once per sequence.
   • Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §2.1
2. Define RAG-Token, and when it helps.
   Each output token can come from a different document: \(p(y \mid x) \approx \prod_{i=1}^{N} \sum_{z \in \text{top-}k} p_\eta(z \mid x)\, p_\theta(y_i \mid x, z, y_{1:i-1})\). It lets the generator combine content from several documents in one answer; for classification (one-token targets) the two variants coincide.
   • Lewis et al., 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §2.1