The Mathematics of Statistical Machine Translation: Parameter Estimation

4.4 Model 3

1. What are Model 3’s parameters?
   Fertility \(n(\phi \mid e)\), translation \(t(f \mid e)\), distortion \(d(j \mid i, m, l)\) (French position given English position and lengths), and \(p_0, p_1\) for the empty cept.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.4
2. How does Model 3 generate words from the empty cept?
   After the real words are generated, each of them may require an extraneous word with probability \(p_1\), connected to \(e_0\); so the number of empty-cept words \(\phi_0\) grows with sentence length, and these words are spread uniformly over the remaining positions.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.4
3. Why can’t Model 3 be trained with exact EM, and what is done instead?
   Fertility prevents exchanging the sum over alignments with the product, so expectations are summed over a small set of probable alignments: start from the best Model 2 alignment, hill-climb by moves (change one \(a_j\)) and swaps (exchange two), and include the neighbours of the result (plus “pegged” variants for each connection).
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.4