Toolformer: Language Models Can Teach Themselves to Use Tools
Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “4.3 Language Modeling” · whole deck
4.3 Language Modeling
- Did learning tool use hurt Toolformer’s language modeling?No: with API calls disabled, its perplexity on WikiText and held-out CCNet was about the same as GPT-J fine-tuned on the same data without calls.
- Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §4.3
Source
- Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/d842425e4bf79ba039352da0f658a906-Abstract-Conference.html