Robust Speech Recognition via Large-Scale Weak Supervision
Radford et al., 2023, Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202)
Subscriber deck — free during early access
1 cards in “4.1 Dataset Scaling” · whole deck
4.1 Dataset Scaling
- How did Whisper’s performance scale with dataset size?Every increase helped, but with diminishing returns: English WER improved quickly up to 13,000 hours and then slowed, gaining only about 1 point from the final 12.5× increase; translation was near zero below 7,000 hours, then improved roughly log-linearly up to 54,000 hours.
- Radford et al., 2023, Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §4.1
Source
- Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, pp. 28492–28518. https://proceedings.mlr.press/v202/radford23a.html