learn.ifs.name

Toolformer: Language Models Can Teach Themselves to Use Tools

Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))

Subscriber deck — free during early access

1 cards in “4.1 Experimental Setup” · whole deck

4.1 Experimental Setup

  1. What models did Schick et al. compare to isolate the effect of tools?
    GPT-J (6.7B); GPT-J fine-tuned on the same CCNet subset without API calls; Toolformer; Toolformer with API calls disabled at decoding; and much larger OPT-66B and GPT-3 175B, all prompted zero-shot without task-specific examples of tool use.
    • Schick et al., 2023, Toolformer (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §4.1

Source