learn.ifs.name

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

Asai et al., 2024, Self-RAG (The Twelfth International Conference on Learning Representations (ICLR 2024))

Subscriber deck — free during early access

2 cards in “5.1 Main Results” · whole deck

5.1 Main Results

  1. How did Self-RAG compare with other models?
    Across six tasks (fact verification, multiple-choice reasoning, open-domain QA, biographies, long-form QA with citations) it beat other open models with and without retrieval, and outperformed ChatGPT on several (e.g. PubHealth, PopQA, biographies), even surpassing it in citation precision on ASQA.
    • Asai et al., 2024, Self-RAG (The Twelfth International Conference on Learning Representations (ICLR 2024)), §5.1
  2. Why did standard retrieval-augmented baselines help little on some tasks?
    On tasks where the answer can’t simply be copied from a passage, such as PubHealth fact checking and ARC-Challenge reasoning, adding retrieved documents barely improved them, and most such baselines also struggled to cite accurately.
    • Asai et al., 2024, Self-RAG (The Twelfth International Conference on Learning Representations (ICLR 2024)), §5.1

Source

  • Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. In The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2310.11511