Robust Speech Recognition via Large-Scale Weak Supervision
Radford et al., 2023, Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202)
Subscriber deck — free during early access
1 cards in “4.3 Multitask and Multilingual Transfer” · whole deck
4.3 Multitask and Multilingual Transfer
- Does training on many tasks and languages hurt English speech recognition?For small models, yes: joint models underperformed English-only models trained with the same compute (negative transfer). But joint models scaled better, and the largest ones outperformed English-only training, showing positive transfer.
- Radford et al., 2023, Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §4.3
Source
- Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, pp. 28492–28518. https://proceedings.mlr.press/v202/radford23a.html