learn.ifs.name

Scaling Neural Machine Translation to 200 Languages

Team, 2024, Scaling neural machine translation to 200 languages (Nature, vol. 630)

Subscriber deck — free during early access

3 cards in “Automatically creating translation training data” · whole deck

Automatically creating translation training data

  1. Why was language identification a key problem for NLLB?
    Collecting monolingual web data for 200 languages needs accurate language ID, which is far from solved on web text: domains differ by language, similar languages are confused, and classifiers trained on found data conflate domain with language. NLLB tuned a fastText model on FLORES-200 plus handwritten rules.
  2. How did NLLB mine parallel sentences from monolingual web data?
    It embedded sentences of all languages in a shared multilingual space (LASER3) and treated pairs with high similarity as translations. New languages were added by teacher–student distillation into the existing LASER space, so previously embedded data needn’t be re-encoded.
    shared multilingual embedding space (LASER3)pairpairpairno match● lang A● lang B
    Sentences whose embeddings are close across languages are mined as parallel data (illustrative)
  3. How much did bitext mining contribute?
    Over 1.1 billion new sentence pairs for 148 languages; together with back-translation, this improved translation of very low-resource languages into English by 12.5 chrF++.

Source