learn.ifs.name

Tülu 3: Pushing Frontiers in Open Language Model Post-Training

Lambert et al., 2024, Tülu 3 (arXiv preprint arXiv:2411.15124)

Subscriber deck — free during early access

1 cards in “5.3 Key Findings of Data Ablations” · whole deck

5.3 Key Findings of Data Ablations

  1. What did Tülu 3’s ablations find about preference data?
    More unique prompts helped, while duplicating prompts with extra responses did not; prompts unused in SFT beat reused ones (a mix was best); and including on-policy responses from the SFT model improved results over purely off-policy data.
    • Lambert et al., 2024, Tülu 3 (arXiv preprint arXiv:2411.15124), §5.3

Source

  • Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi and Hannaneh Hajishirzi. 2024. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. In arXiv preprint arXiv:2411.15124. https://arxiv.org/abs/2411.15124