Tülu 3: Pushing Frontiers in Open Language Model Post-Training
Lambert et al., 2024, Tülu 3 (arXiv preprint arXiv:2411.15124)
Subscriber deck — free during early access
1 cards in “4.3.2 Batch Aggregation” · whole deck
4.3.2 Batch Aggregation
- What loss-aggregation pitfall did Tülu 3 run into during SFT, and how did they fix it?Averaging the loss over tokens within each micro-batch, then over micro-batches in gradient accumulation or across devices, weights examples differently from averaging over all tokens at once, so results depended on the hardware setup. They switched to a summed loss (with a lower learning rate, 5×10⁻⁶), weighting every token equally.
- Lambert et al., 2024, Tülu 3 (arXiv preprint arXiv:2411.15124), §4.3.2
Source
- Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi and Hannaneh Hajishirzi. 2024. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. In arXiv preprint arXiv:2411.15124. https://arxiv.org/abs/2411.15124