Issue #54 – Pivot-based Transfer Learning for Neural MT
03 Oct19
Issue #54 – Pivot-based Transfer Learning for Neural MT
Author: Dr. Rohit Gupta, Sr. Machine Translation Scientist @ Iconic
Pivot-based Transfer Learning for Neural MT between Non-English Languages
Neural MT for many non-English languages is still a challenge because of the unavailability of direct parallel data between these languages. In general, translation between non-English languages, e.g. French to German, is usually done with pivoting through English, i.e., translating French (source) input to English (pivot) and English (pivot) into German. However, pivoting requires doubled decoding time and the translation errors are propagated or expanded via the two-step process.
One approach to build such a direct system is to train a multilingual engine involving all three languages. In this blog post, we will have a look at pivot based transfer learning with pre-training, a recently proposed approach by Kim et al. (2019), which performed better compared to the multilingual model training.
Pivot Based Transfer Learning
- Pre-train a source-pivot model with a source-pivot parallel corpus and a pivot-target model with a pivot-target parallel corpus.
- Initialize the source-target model with the source encoder from the pre-trained source-pivot model and the target decoder from the pre-trained pivot-target model.
- Continue the training
To finish reading, please visit source site