Machine Translation

13.5 Low-resource translation

1. What is data augmentation?
   Adding synthetic training data generated from the current natural data to address insufficient training data.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, p. 291
2. Describe backtranslation.
   To improve source→target MT with a small bitext and plenty of target-language monolingual text: train a target→source system on the bitext, translate the monolingual target text into the source language, and add these synthetic pairs (MT source, natural target) to the training data.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, p. 291
3. What parameters does backtranslation have, and how well does it work?
   How the synthetic source is generated (greedy, beam search, sampling) and the ratio of backtranslated to natural data (natural bitext can be upsampled). It works surprisingly well: roughly two-thirds of the gain of the same amount of natural bitext.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, pp. 291–292
4. How does a multilingual translation model know which languages to translate between?
   Special tokens: a source-language token \(l_s\) for the encoder and a target-language token \(l_t\) for the decoder: \(h = \text{encoder}(x, l_s)\), \(y_{i+1} = \text{decoder}(h, l_t, y_{1:i})\).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, p. 292
5. How can multilingual MT help low-resource languages?
   It can draw on a similar higher-resource language in the training data, e.g. understanding a Galician word through Spanish.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, p. 292
6. What sociotechnical problems affect MT for low-resource languages?
   Native speakers are often not involved as curators, technologists or evaluators; audits found many multilingual corpora mostly of unacceptable quality (boilerplate, wrong translations); and resources focus on pairs involving English.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, p. 292
7. Why did a participatory African MT project recommend evaluating by post-editing?
   Having labelers minimally edit MT output and measuring the distance is simpler to train evaluators for, and captures true errors rather than legitimate linguistic variation.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 13, §13.5, pp. 292–293