Self-Instruct: Aligning Language Models with Self-Generated Instructions

1 Introduction

1. What bottleneck of instruction tuning does Self-Instruct address?
   Instruction-tuned models depend on human-written instruction data, which is costly and limited in quantity, diversity and creativity (mostly popular NLP tasks), limiting how well the tuned model generalizes.
   • Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §1, p. 13484
2. What is Self-Instruct in one sentence?
   An almost annotation-free, iterative bootstrapping method in which a vanilla pretrained language model generates new task instructions and input–output instances from a small seed set, filters them, and is then fine-tuned on its own generated data.
   • Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §1, p. 13485
3. What were Self-Instruct’s headline results with GPT-3?
   From 175 seed tasks, vanilla GPT-3 generated about 52K instructions with 82K instances; fine-tuning GPT-3 on them improved SuperNaturalInstructions by 33 points absolute, nearly matching InstructGPT-001 (trained on private user data and human labels), and left only a 5% gap to it in human evaluation on novel tasks.
   • Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §1, p. 13485