Issue #87 – YiSi – A Unified Semantic MT Quality Evaluation and Estimation Metric

25 Jun20

Author: Dr. Karin Sim, Machine Translation Scientist @ Iconic

Introduction

Automatic evaluation is an issue that has long troubled machine translation (MT): how do we evaluate how good the MT output is? Traditionally, BLEU has been the “go to”, as it is simple to use across language pairs. However, it is overly simplistic, evaluating string matches to a single reference translation. More sophisticated metrics have come on the scene, including chfF (Popović, 2015), TER (Snover et al, 2006), and METEOR (Banerjee and Lavie, 2005). None of them attempt to evaluate the extent to which the meaning of the source is transferred to the target text. The first real attempt to incorporate a semantic element into automatic evaluation was MEANT (Lo and Wu, 2011). However, the fact that it requires additional linguistic resources made it less easy to use widely. YiSi (Lo, 2019) builds on that work, offering a range of flavours based on the level of resources available for that language pair.
To finish reading, please visit source site