Toolformer: Language Models Can Teach Themselves to Use Tools
Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “4.4 Scaling Laws” · whole deck
4.4 Scaling Laws
- How did the ability to use tools depend on model size?Applied to GPT-2 models from 124M to 1.6B parameters, useful tool use emerged only around 775M; smaller models did about the same with and without tools. Larger models got better without tools, but also better at using them, so the gap persisted.
- Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §4.4
Source
- Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/d842425e4bf79ba039352da0f658a906-Abstract-Conference.html