learn.ifs.name

LLaMA: Open and Efficient Foundation Language Models

Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971)

Subscriber deck — free during early access

1 cards in “3.2 Closed-book Question Answering” · whole deck

3.2 Closed-book Question Answering

  1. What did LLaMA achieve on closed-book question answering, and why is the 13B result notable?
    LLaMA-65B reached state of the art on Natural Questions and TriviaQA in zero- and few-shot settings; LLaMA-13B was competitive with GPT-3 and Chinchilla while being 5–10× smaller and runnable on a single V100 GPU.
    • Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971), §3.2

Source

  • Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. In arXiv preprint arXiv:2302.13971. https://arxiv.org/abs/2302.13971