Toolformer: Language Models Can Teach Themselves to Use Tools

2 Approach

1. How are API calls represented in Toolformer’s text?
   As linearized sequences \(e(c, r) = \) <API> \(a_c(i_c) \to r\) </API>, with API name \(a_c\), input \(i_c\) and result \(r\), inserted inline in the text; in practice the special tokens are written as “[”, “]” and “->”, so the vocabulary needn’t change.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
2. Outline the steps of Toolformer’s data generation.
   (1) With a few-shot prompt per tool, let the model itself propose API calls at positions where it assigns high probability to starting one; (2) execute the calls; (3) keep only calls whose results make the following tokens easier to predict; (4) merge the kept calls into the original texts and fine-tune the model on this augmented dataset.

   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
3. What is Toolformer’s filtering criterion for API calls?
   Compare the weighted loss on the following tokens with the call and its result as prefix, \(L_i^+\), against the smaller of the losses with no call and with the call but no result, \(L_i^-\). Keep the call only if \(L_i^- - L_i^+ \ge \tau_f\): input and output together must reduce the loss by at least a threshold.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
4. Why does Toolformer compare against “call without result” as well as “no call”?
   To make sure it is the tool’s output that helps: a call whose input alone (e.g. a question restating useful words) lowers the loss would otherwise be kept even though the tool contributes nothing.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
5. Why does fine-tuning on Toolformer’s augmented data preserve the model’s general abilities?
   The augmented dataset contains exactly the same texts as the original (here a subset of CCNet, like the pretraining data), with API calls added only where they help, so the model sees the same content while learning when and how to call tools.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
6. How does Toolformer call a tool during generation?
   It decodes normally until it produces the “→” token, then decoding pauses, the API is called, and its response and the closing </API> token are inserted before decoding continues.
   • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2