Neural Machine Translation of Rare Words with Subword Units
Sennrich et al., 2016, Neural Machine Translation of Rare Words with Subword Units (Proceedings of the 54th Annual Meeting of ACL (Volume 1: Long Papers))
Subscriber deck — free during early access
1 cards in “4.1 Subword statistics” · whole deck
4.1 Subword statistics
- How do segmentation schemes trade off vocabulary size and sequence length?Characters give a tiny vocabulary but 5.5× longer text; character n-grams and shortlists sit in between. Compound splitting, Morfessor and hyphenation reduce the vocabulary only moderately and leave unknown words. BPE (63k types) barely lengthens the text (112M vs 100M tokens) and leaves no unknown symbols.
- Sennrich et al., 2016, Neural Machine Translation of Rare Words with Subword Units (Proceedings of the 54th Annual Meeting of ACL (Volume 1: Long Papers)), §4.1, pp. 1719–1720
Source
- Rico Sennrich, Barry Haddow and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/P16-1162 https://aclanthology.org/P16-1162/