Toolformer: Language Models Can Teach Themselves to Use Tools

1 Introduction

1. What paradox motivates Toolformer?
   Large language models solve new tasks from a few examples, yet struggle with basic things that small specialized tools do well: precise arithmetic, factual lookup, up-to-date information, low-resource languages and awareness of the current date.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §1
2. What two requirements did Schick et al. set for learning tool use?
   It should be self-supervised, without large amounts of human annotation (which is costly, and what humans find useful may differ from what helps the model); and the model should keep its generality and decide for itself when and how to use which tool, not only in task-specific settings.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §1