Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers))
Subscriber deck — free during early access
1 cards in “8 Limitations” · whole deck
8 Limitations
- What limitations of Self-Instruct do Wang et al. acknowledge?It inherits language-model weaknesses on tail phenomena, so gains may favour frequent kinds of tasks; it may work best with large models, raising access barriers; and the iterative loop could amplify social biases, as seen in its difficulty producing balanced labels.
- Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §8, p. 13493
Source
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2023.acl-long.754 https://aclanthology.org/2023.acl-long.754/