learn.ifs.name

LLaMA: Open and Efficient Foundation Language Models

Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971)

Subscriber deck — free during early access

1 cards in “3.4 Mathematical reasoning” · whole deck

3.4 Mathematical reasoning

  1. How did LLaMA do on mathematical reasoning without math fine-tuning?
    On GSM8k LLaMA-65B outperformed Minerva-62B, a PaLM model fine-tuned on 38.5B tokens of math and arXiv data, though LLaMA saw no special math data.
    • Touvron et al., 2023, LLaMA (arXiv preprint arXiv:2302.13971), §3.4

Source

  • Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. In arXiv preprint arXiv:2302.13971. https://arxiv.org/abs/2302.13971