learn.ifs.name

BLEU: a Method for Automatic Evaluation of Machine Translation

Papineni et al., 2002, BLEU (Proceedings of the 40th Annual Meeting of ACL)

Subscriber deck — free during early access

1 cards in “5 BLEU vs the human evaluation” · whole deck

5 BLEU vs the human evaluation

  1. How well did BLEU correlate with human judgments?
    Linear regression of human scores on BLEU (two references, five systems) gave correlation 0.99 with monolingual judges and 0.96 with bilingual judges; BLEU even captured the small difference between two close MT systems and the large gap between MT and human translations.
    • Papineni et al., 2002, BLEU (Proceedings of the 40th Annual Meeting of ACL), §5

Source

  • Kishore Papineni, Salim Roukos, Todd Ward and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318. https://aclanthology.org/P02-1040/