March 20, 2020 machine translation, NLP Leave a comment

Issue #74 – Transfer Learning for Neural Machine Translation

20 Mar20 Issue #74 – Transfer Learning for Neural Machine Translation Author: Dr. Chao-Hong Liu, Machine Translation Scientist @ Iconic Building machine translation (MT) for low-resource languages is a challenging task. This is especially true when training using neural MT (NMT) methods that require a comparatively larger corpus of parallel data. In this post, we review the work done by Zoph et al. (2016) on training NMT systems for low-resource languages using transfer learning. Transfer Learning The idea of transfer […]

Issue #73 – Mixed Multi-Head Self-Attention for Neural MT

12 Mar20 Issue #73 – Mixed Multi-Head Self-Attention for Neural MT Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic Self-attention is a key component of the Transformer, a state-of-the-art neural machine translation architecture. In the Transformer, self-attention is divided into multiple heads to allow the system to independently attend to information from different representation subspaces. Recently it has been shown that some redundancy occurs in the multiple heads. In this post, we take a look at approaches which ensure […]

February 13, 2020 NLP, Paraphrases in Multilingual Neural MT Leave a comment

Issue #69 – Using Paraphrases in Multilingual Neural MT

13 Feb20 Issue #69 – Using Paraphrases in Multilingual Neural MT Author: Dr. Chao-Hong Liu, Machine Translation Scientist @ Iconic Paraphrasing is common in human languages, as a way to talk about the same thing in different ways. There are many possible sentences that could be used to express the same meaning. From an MT perspective, we wanted to train systems that could not only translate sentences that bear similar meaning in one language into a sentence in another language, […]

February 7, 2020 BERT in Neural MT, NLP Leave a comment

Issue #68 – Incorporating BERT in Neural MT

07 Feb20 Issue #68 – Incorporating BERT in Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic BERT (Bidirectional Encoder Representations from Transformers) has shown impressive results in various Natural Language Processing (NLP) tasks. However, how to effectively apply BERT in Neural MT has not been fully explored. In general, BERT is used as fine-tuning for downstream NLP tasks. For Neural MT, a pre-trained BERT model is used to initialise the encoder in an encoder-decoder architecture. In this post we […]

January 30, 2020 NLP, Unsupervised Adaptation of NMT with Iterative Back-Translation Leave a comment

Issue #67 – Unsupervised Adaptation of Neural MT with Iterative Back-Translation

30 Jan20 Issue #67 – Unsupervised Adaptation of Neural MT with Iterative Back-Translation Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic The most popular domain adaptation approach, when some in-domain data are available, is to fine-tune the training of the generic model with the in-domain corpus. When no parallel in-domain data are available, the most popular approach is back-translation, which consists of translating monolingual target in-domain data into the source language and use it as training corpus. In this […]

January 23, 2020 Low-Resource Languages, NLP Leave a comment

Issue #66 – Neural Machine Translation Strategies for Low-Resource Languages

23 Jan20 Issue #66 – Neural Machine Translation Strategies for Low-Resource Languages This week we are pleased to welcome the newest member to our scientific team, Dr. Chao-Hong Liu. In this, his first post with us, he’ll give his views on two specific MT strategies, namely, pivot MT and zero-shot MT. While we have covered these topics in previous ‘Neural MT Weekly’ blog posts (Issue #54, Issue #40), these are topics that Chao-Hong has recently worked on prior to joining […]

December 13, 2019 Byte-level subwords, NLP Leave a comment

Issue #64 – Neural Machine Translation with Byte-Level Subwords

13 Dec19 Issue #64 – Neural Machine Translation with Byte-Level Subwords Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic In order to limit vocabulary, most neural machine translation engines are based on subwords. In some settings, character-based systems are even better (see issue #60). However, rare characters in noisy data or character-based languages can unnecessarily take up vocabulary slots and limit its compactness. In this post we take a look at an alternative, proposed by Wang et al. (2019), […]

December 5, 2019 Neuron Interaction Based Representation Composition, NLP Leave a comment

Issue #63 – Neuron Interaction Based Representation Composition for Neural Machine Translation

05 Dec19 Issue #63 – Neuron Interaction Based Representation Composition for Neural Machine Translation Author: Dr. Rohit Gupta, Sr. Machine Translation Scientist @ Iconic Transformer models are state of the art in Neural Machine Translation. In this blog post, we will take a look at a recently proposed approach by Li et al (2019) which further improves upon the transformer model by modeling more neuron interactions. Li et al (2019) claim that their approach models better encoder representation and captures semantic […]

November 28, 2019 Domain Differential Adaptation, NLP Leave a comment

Issue #62 – Domain Differential Adaptation for Neural MT

28 Nov19 Issue #62 – Domain Differential Adaptation for Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic Neural MT models are data hungry and domain sensitive, and it is nearly impossible to obtain a good amount ( >1M segments) of training data for every domain we are interested in. One common strategy is to align the statistics of the source and target domain, but the drawback of this approach is that the statistics of the different domains are inherently […]

November 21, 2019 Context-aware Monolingual Repail for Neural MT, NLP Leave a comment

Issue #61 – Context-Aware Monolingual Repair for Neural Machine Translation

21 Nov19 Issue #61 – Context-Aware Monolingual Repair for Neural Machine Translation Author: Dr. Rohit Gupta, Sr. Machine Translation Scientist @ Iconic In issue #15 and issue #39 we looked at various approaches for document level translation. In this blog post, we will look at another approach proposed by Voita et. al (2019a) to capture context information. This approach is unique in the sense that it utilizes only target monolingual data to improve the discourse phenomenon (deixis, ellipsis, lexical cohesion, ambiguity, […]

« 1 … 964 965 966 967 968 … 972 »