The Mathematics of Statistical Machine Translation: Parameter Estimation

4.5 Deficiency

1. What does it mean that a model is deficient?
   It wastes probability on objects that aren’t French strings. Models 3 and 4 place each word independently, so several words can land in one position and others stay empty (“generalized strings”). It’s the price of simplicity.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.5
2. Why are Models 1 and 2 “spiritually deficient”?
   They give the same probability to “Je n’ai pas de stylo” and the scrambled “Je pas ai ne de stylo” as translations of “I do not have a pen”. This hardly matters, since the models are only used to predict English given real French and need be accurate only up to a constant factor.
   • Brown et al., 1993, The Mathematics of Statistical Machine Translation (Computational Linguistics), §4.5