learn.ifs.name

LLaMA: Open and Efficient Foundation Language Models

Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971)

Subscriber deck — free during early access

1 cards in “4 Instruction Finetuning” · whole deck

4 Instruction Finetuning

  1. What did brief instruction fine-tuning do for LLaMA-65B?
    A single fine-tuning run following Flan’s protocol (LLaMA-I) raised MMLU from 63.4 to 68.9, beating other moderate-sized instruction-tuned models, though still well below code-davinci-002’s 77.4.
    • Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971), §4

Source

  • Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. In arXiv preprint arXiv:2302.13971. https://arxiv.org/abs/2302.13971