June 27, 2019 NLP, Overcorrection Recovery Leave a comment

Issue #43 – Improving Overcorrection Recovery in Neural MT

27 Jun19 Issue #43 – Improving Overcorrection Recovery in Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic In Neural MT, at training time, the model predicts the current word with the ground truth word (previous word in the sequence) as a context, while at inference time it has to generate the complete sequence. This discrepancy in training and inference often leads to an accumulation of errors in the translation process, resulting in out-of-context translations. In this post we’ll discuss […]

June 13, 2019 Deep Transformer Models, NLP Leave a comment

Issue #41 – Deep Transformer Models for Neural MT

13 Jun19 Issue #41 – Deep Transformer Models for Neural MT Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic The Transformer is a state-of-the-art Neural MT model, as we covered previously in Issue #32. So what happens when something works well with neural networks? We try to go wider and deeper! There are two research directions that look promising to enhance the Transformer model: building wider networks by increasing the size of word representation and attention vectors, or building […]

June 6, 2019 Neural MT, NLP Leave a comment

Issue #40 – Consistency by Agreement in Zero-shot Neural MT

06 Jun19 Issue #40 – Consistency by Agreement in Zero-shot Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic In two of our earlier posts (Issues #6 and #37), we discussed the zero-shot approach to Neural MT – learning to translate from source to target without seeing even a single example of the language pair directly. In Neural MT, the zero-shot training is achieved using multilingual architecture (Johnson et al. 2017) – a single NMT engine that can translate between […]