Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers))
Subscriber deck — free during early access
1 cards in “4.4 Experiment 2: Generalization to User-oriented Instructions on Novel Tasks” · whole deck
4.4 Experiment 2: Generalization to User-oriented Instructions on Novel Tasks
- Why did Wang et al. write 252 new user-oriented instructions, and how were they evaluated?SuperNI tasks come from research datasets skewed toward classification; the new set covers practical uses (email, social media, programming, entertainment) in varied formats. Since many outputs (programs, logic) need expertise, the instructions’ authors rated outputs on a four-level scale from valid to irrelevant.
- Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §4.4, pp. 13489–13490
Source
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2023.acl-long.754 https://aclanthology.org/2023.acl-long.754/