learn.ifs.name

Self-Instruct: Aligning Language Models with Self-Generated Instructions

Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers))

Subscriber deck — free during early access

1 cards in “2.1 Defining Instruction Data” · whole deck

2.1 Defining Instruction Data

  1. How does Self-Instruct define a task, and why can the input be empty?
    A task \(t\) is an instruction \(I_t\) plus one or more instances \((X_{t,i}, Y_{t,i})\) with \(M(I_t, X_{t,i}) = Y_{t,i}\). The boundary between instruction and input is fluid (“write an essay about school safety” vs “write an essay about the following topic” + “school safety”), so instructions without input are allowed for format diversity.
    • Wang et al., 2023, Self-Instruct (Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers)), §2.1, p. 13486

Source

  • Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/2023.acl-long.754 https://aclanthology.org/2023.acl-long.754/