Statistical Phrase-Based Translation

2.1 Model

1. What is the overall form of the phrase-based model in Koehn et al. (2003)?
   A noisy-channel model with a length bonus: \(\mathbf{e}_{\text{best}} = \arg\max_{\mathbf{e}} p(\mathbf{f} \mid \mathbf{e})\, p_{\text{LM}}(\mathbf{e})\, \omega^{\text{length}(\mathbf{e})}\), with a trigram language model \(p_{\text{LM}}\). The factor \(\omega\), usually above 1, biases towards longer output to calibrate length.
   • Koehn et al., 2003, Statistical Phrase-Based Translation (Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of ACL (HLT-NAACL 2003)), §2.1, pp. 48–49
2. How does the phrase model decompose \(p(\bar f_1^I \mid \bar e_1^I)\)?
   The foreign sentence is segmented into \(I\) phrases (all segmentations equally likely); each is translated independently and the English phrases may be reordered: \(p(\bar f_1^I \mid \bar e_1^I) = \prod_{i=1}^{I} \phi(\bar f_i \mid \bar e_i)\, d(a_i - b_{i-1})\), with phrase translation probabilities \(\phi\) and a distortion model \(d\).
   • Koehn et al., 2003, Statistical Phrase-Based Translation (Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of ACL (HLT-NAACL 2003)), §2.1, pp. 48–49
3. What do \(a_i\) and \(b_{i-1}\) denote in the distortion term \(d(a_i - b_{i-1})\), and what is the simple distortion model?
   \(a_i\) is the start position of the foreign phrase translated into the \(i\)th English phrase, \(b_{i-1}\) the end position of the foreign phrase translated into the \((i-1)\)th one. A simple choice is \(d(a_i - b_{i-1}) = \alpha^{|a_i - b_{i-1} - 1|}\), which penalizes jumps; monotone translation (\(a_i = b_{i-1} + 1\)) costs nothing.
   • Koehn et al., 2003, Statistical Phrase-Based Translation (Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of ACL (HLT-NAACL 2003)), §2.1, pp. 48–49
4. Why is the phrase translation model \(\phi(\bar f \mid \bar e)\) “backwards”, from foreign given English?
   Because of Bayes’ rule in the noisy-channel formulation: to translate \(\mathbf{f}\) into English we maximize \(p(\mathbf{f} \mid \mathbf{e})\, p(\mathbf{e})\), so the translation direction is inverted from the modelling standpoint.
   • Koehn et al., 2003, Statistical Phrase-Based Translation (Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of ACL (HLT-NAACL 2003)), §2.1, pp. 48–49