learn.ifs.name

Self-Instruct: Aligning Language Models with Self-Generated Instructions

Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers))

Subscriber deck — free during early access

1 cards in “5 Related Work” · whole deck

5 Related Work

  1. How does Self-Instruct differ from Unnatural Instructions and from knowledge distillation?
    Unnatural Instructions seeded from SuperNI tasks and generated with already instruction-tuned InstructGPT-002, distilling from it; Self-Instruct uses only a vanilla model. Unlike usual distillation, source and target are the same model, and what is distilled is whole tasks (instructions with instances).
    • Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §5, pp. 13491–13492

Source

  • Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2023.acl-long.754 https://aclanthology.org/2023.acl-long.754/