Machine Translation Weekly 61: Decoding and diversity
This week I will comment on a short paper from Carnegie Mellon University and
Amazon that shows a simple analysis of the diversity of machine translation
outputs. The title of the paper is Decoding and Diversity in Machine
Translation and it will be presented at the
Resistance AI
Workshop at
NeuRIPS 2020 (what a name for a workshop).
The main thing that the paper shows that is the translation quality measured in
terms of BLEU score strongly negatively correlates with some desirable
properties of machine translation that can be described with an umbrella term
output diversity. This is even more disturbing when we take into account that
the systems with high BLEU scores typically end up as indistinguishable from
human translation in the WMT evaluation campaigns, even though