January 5, 2021 ABSTRACTIVE TEXT SUMMARIZATION, LANGUAGE MODELLING, machine translation, Neural MT, NMT

Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers

The advent of the Transformer can arguably be described as a driving force behind many of the recent advances in natural language processing. However, despite their sizeable performance improvements, as recently shown, the model is severely over-parameterized, being parameter inefficient and computationally expensive to train… Inspired by the success of parameter-sharing in pretrained deep contextualized word representation encoders, we explore parameter-sharing methods in Transformers, with a specific focus on encoder-decoder models for sequence-to-sequence tasks such as neural machine translation. We […]

January 4, 2021 Machine Learning

Semi-Supervised Learning With Label Spreading

Semi-supervised learning refers to algorithms that attempt to make use of both labeled and unlabeled training data. Semi-supervised learning algorithms are unlike supervised learning algorithms that are only able to learn from labeled training data. A popular approach to semi-supervised learning is to create a graph that connects examples in the training dataset and propagates known labels through the edges of the graph to label unlabeled examples. An example of this approach to semi-supervised learning is the label spreading algorithm […]

January 2, 2021 Python

How to Use Global and Nonlocal Variables in Python

Introduction In this article we’ll be taking a look at Global and Non-Local Variables in Python and how you to use them to avoid issues when writing code. We’ll be starting off with a brief primer on variable scopes before we launch into the how and why of using global and non-local variables in your own functions. Scopes in Python Before we can get started, we first have to touch on scopes. For those of you who are less familiar, […]

January 2, 2021 LANGUAGE MODELLING, machine translation, Neural MT, NMT, TEXT GENERATION

Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation

Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contains specific words, is a challenging problem… Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them. Methods capable of doing this are called plug-and-play. Recent plug-and-play methods have been successful in constraining small bidirectional language models as well as […]

January 1, 2021 Machine Learning

Semi-Supervised Learning With Label Propagation

Semi-supervised learning refers to algorithms that attempt to make use of both labeled and unlabeled training data. Semi-supervised learning algorithms are unlike supervised learning algorithms that are only able to learn from labeled training data. A popular approach to semi-supervised learning is to create a graph that connects examples in the training dataset and propagate known labels through the edges of the graph to label unlabeled examples. An example of this approach to semi-supervised learning is the label propagation algorithm […]

January 1, 2021 Machine Learning

Multinomial Logistic Regression With Python

Multinomial logistic regression is an extension of logistic regression that adds native support for multi-class classification problems. Logistic regression, by default, is limited to two-class classification problems. Some extensions like one-vs-rest can allow logistic regression to be used for multi-class classification problems, although they require that the classification problem first be transformed into multiple binary classification problems. Instead, the multinomial logistic regression algorithm is an extension to the logistic regression model that involves changing the loss function to cross-entropy loss […]

December 31, 2020 Advanced, Deep Learning, NLP, Project, Python, Semi-supervised, Structured Data, Technique, Text Leave a comment

Fake news classifier on US Election News📰 | LSTM 🈚

Introduction News media has become a channel to pass on the information of what’s happening in the world to the people living. Often people perceive whatever conveyed in the news to be true. There were circumstances where even the news channels acknowledged that their news is not true as they wrote. But some news has a significant impact not only on the people or

December 30, 2020 Python

Ultimate Guide to Heatmaps in Seaborn with Python

Introduction A heatmap is a data visualization technique that uses color to show how a value of interest changes depending on the values of two other variables. For example, you could use a heatmap to understand how air pollution varies according to the time of day across a set of cities. Another, perhaps more rare case of using heatmaps is to observe human behavior – you can create visualizations of how people use social media, how their answers on surveys […]

December 30, 2020 machine translation, Neural MT, NMT

Towards Fully Automated Manga Translation

We tackle the problem of machine translation of manga, Japanese comics. Manga translation involves two important problems in machine translation: context-aware and multimodal translation… Since text and images are mixed up in an unstructured fashion in Manga, obtaining context from the image is essential for manga translation. However, it is still an open problem how to extract context from image and integrate into MT models. In addition, corpus and benchmarks to train and evaluate such model is currently unavailable. In […]

December 29, 2020 machine translation, Neural MT, NMT

Learning Light-Weight Translation Models from Deep Transformer

Recently, deep models have shown tremendous improvements in neural machine translation (NMT). However, systems of this kind are computationally expensive and memory intensive… In this paper, we take a natural step towards learning strong but light-weight NMT systems. We proposed a novel group-permutation based knowledge distillation approach to compressing the deep Transformer model into a shallow model. The experimental results on several benchmarks validate the effectiveness of our method. Our compressed model is 8X shallower than the deep model, with […]

« 1 … 757 758 759 760 761 … 985 »