wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski et al., 2020, wav2vec 2.0 (Advances in Neural Information Processing Systems 33 (NeurIPS 2020))
Subscriber deck — free during early access
1 cards in “3.3 Fine-tuning” · whole deck
3.3 Fine-tuning
- How is wav2vec 2.0 fine-tuned for speech recognition?Add a randomly initialized linear projection on top of the Transformer into the task vocabulary (29 characters plus a word boundary for LibriSpeech), train with a CTC loss, and mask time steps and channels as in SpecAugment, which delays overfitting, especially with very little labelled data. The feature encoder is frozen.
- Baevski et al., 2020, wav2vec 2.0 (Advances in Neural Information Processing Systems 33 (NeurIPS 2020)), §3.3
Source
- Alexei Baevski, Henry Zhou, Abdelrahman Mohamed and Michael Auli. 2020. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). https://papers.nips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html