Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

5.2 Analysis

1. What did Self-RAG’s ablations show?
   Training without retrieved passages or without the critic’s reflection tokens lost much performance; always using the top retrieved passage regardless of relevance (as in standard RAG) hurt PopQA and ASQA; dropping the IsSup score from beam search hurt ASQA.
   • Asai et al., 2024, Self-RAG (The Twelfth International Conference on Learning Representations (ICLR 2024)), §5.2
2. What trade-off does weighting the IsSup score expose?
   Raising its weight increased citation precision (claims backed by evidence) but lowered fluency (MAUVE), since longer, more fluent answers contain more unsupported claims; Self-RAG lets users pick the balance without retraining.
   • Asai et al., 2024, Self-RAG (The Twelfth International Conference on Learning Representations (ICLR 2024)), §5.2