Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers))
Subscriber deck — free during early access
1 cards in “4.3 Experiment 1: Zero-Shot Generalization on SuperNI benchmark” · whole deck
4.3 Experiment 1: Zero-Shot Generalization on SuperNI benchmark
- How did GPT-3 fine-tuned on Self-Instruct data compare with other models on SuperNI?It outperformed vanilla GPT-3 (which mostly produced irrelevant, repetitive text) by 33 points and beat T0 and GPT-3 fine-tuned on T0’s human-written data, nearly matching InstructGPT-001. Models trained on SuperNI’s own training set did better, but adding Self-Instruct data to it still helped.
- Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §4.3, p. 13489
Source
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2023.acl-long.754 https://aclanthology.org/2023.acl-long.754/