The Mathematics of Statistical Machine Translation: Parameter Estimation

4.2 Model 2

1. What does Model 2 add, and why does it remain tractable?
   Alignment probabilities \(a(i \mid j, m, l)\) depending on the positions and the two lengths, so word order matters. The same sum–product interchange still applies: \(\Pr(\mathbf{f} \mid \mathbf{e}) = \epsilon \prod_j \sum_i t(f_j \mid e_i)\, a(i \mid j, m, l)\). Model 1 is Model 2 with \(a\) fixed at \(1/(l+1)\).
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.2
2. How can parameters be transferred from one model to the next?
   Compute alignment probabilities with the old model but collect counts as appropriate for the new one, e.g. initialize Model 2 by using Model 1 to weight alignments while counting position statistics.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.2
3. What is the Viterbi alignment, and how is it found for Model 2?
   The most probable alignment \(V(\mathbf{f} \mid \mathbf{e})\). For Models 1 and 2, choose each \(a_j\) independently to maximize \(t(f_j \mid e_{a_j})\, a(a_j \mid j, m, l)\); for general models no practical exact algorithm is known.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.2