The Mathematics of Statistical Machine Translation: Parameter Estimation

2 Statistical translation

1. What did Warren Weaver suggest in 1949, and why did it fail then?
   Applying statistical and cryptanalytic techniques from communication theory to machine translation. Such efforts were soon abandoned for philosophical and theoretical reasons, but with computers “of a piece with today’s digital watch” they were doomed to computational starvation anyway.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §2
2. State the Fundamental Equation of Machine Translation.
   To translate French \(\mathbf{f}\), choose \(\hat{\mathbf{e}} = \arg\max_{\mathbf{e}} \Pr(\mathbf{e}) \Pr(\mathbf{f} \mid \mathbf{e})\), from Bayes’ theorem, since \(\Pr(\mathbf{f})\) doesn’t depend on \(\mathbf{e}\). The view: a French speaker had an English sentence in mind and translated it; find that sentence.

   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §2
3. Why is the fundamental equation “fanciful” as a model of human translation but formally adequate?
   No translator rifles through all English passages computing products of probabilities; intuitively they understand the French and then express its meaning. But formally \(\Pr(\mathbf{f} \mid \mathbf{e})\) is just an enormous table, and with the right values translations of arbitrarily high quality are possible. The real question is empirical: can good enough approximations be built?
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §2
4. Why estimate \(\Pr(\mathbf{e})\Pr(\mathbf{f} \mid \mathbf{e})\) instead of \(\Pr(\mathbf{e} \mid \mathbf{f})\) directly?
   The translation models are “prodigal”, spreading most probability over ill-formed strings, which doesn’t matter for \(\Pr(\mathbf{f} \mid \mathbf{e})\) (a constant factor doesn’t change the argmax). The two factors cooperate: the translation model favours English with the right words roughly in the right places, the language model favours well-formed English. Reversing a translation model would put too little probability on well-formed English.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §2