BLEU: a Method for Automatic Evaluation of Machine Translation

2.2 Combining the modified n-gram precisions

1. Why does BLEU combine n-gram precisions with a geometric mean rather than an arithmetic one?
   Modified precision decays roughly exponentially with \(n\), so a linear average would be dominated by low orders; averaging logarithms (equivalently a geometric mean) accounts for the decay, and correlated slightly better with human judgments.
   • Papineni et al., 2002, BLEU (Proceedings of the 40th Annual Meeting of ACL), §2.2
2. What is the downside of the geometric mean in BLEU?
   It is harsh: if any modified precision is zero, the whole score is zero, which the authors expected to be very rare on test corpora of reasonable size with \(N \le 4\) (but is common for single sentences).
   • Papineni et al., 2002, BLEU (Proceedings of the 40th Annual Meeting of ACL), §2.2