Machine Translation Weekly 97: Multilingual and Non-autoregressive MT at the same time
Multilingual machine translation models look very promising, especially for
low-resource languages that can benefit from similar patterns in similar
languages. A new preprint with authors from the University of Maryland and
Google Research studies how these results transfer to non-autoregressive
machine translation models. The title of the paper is Can Multilinguality
benefit Non-autoregressive Machine
Translation?. Spoiler: it is not as good as
it might seem.
The paper tries to answer two questions: First, is it better to use a
multilingual teacher model or bilingual teacher models? Second, how does the
positive and negative transfer differ in the case of autoregressive and
non-autoregressive models? The negative transfer happens if unrelated
languages share a model, so the model has a limited capacity. The positive
transfer happens if the languages are related