The Mathematics of Statistical Machine Translation: Parameter Estimation

6 Better translation models

1. Why does deficiency matter for decoding after all?
   Without long-range constraints, probability on well-formed French shrinks exponentially with English length, so \(\Pr(\mathbf{e})\Pr(\mathbf{f} \mid \mathbf{e})\) unduly favours short English. Multiplying by \(c^l\) for an empirical \(c\) treats the symptom, not the disease.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §6
2. What is Viterbi training, and why does it converge?
   Re-estimate parameters from the single most probable alignment of each pair instead of expectations. Each step makes the current Viterbi alignments more probable, and probability can’t exceed 1, so it converges, in a finite number of steps since there are finitely many alignments.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §6
3. Why is going beyond one-word cepts difficult, and how would multi-word cepts be chosen?
   There are about 42,000 one-word cepts but over 1.7 billion two-word and 74 trillion three-word candidates. Promote a sequence only when its translations differ from what its words predict: high and dry → échoué or en plan, not haut et sec.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §6
4. What morphology problem did the authors plan to address?
   Every letter sequence is a separate word, so the 5 forms of eat must be linked to the 39 forms of manger, most never seen. They planned invertible inflectional analysis (root + ending, e.g. mange → manger + 13spres) to cut the French vocabulary by about half.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §6