January 2, 2021 LANGUAGE MODELLING, machine translation, Neural MT, NMT, TEXT GENERATION

Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation

Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contains specific words, is a challenging problem… Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them. Methods capable of doing this are called plug-and-play. Recent plug-and-play methods have been successful in constraining small bidirectional language models as well as […]

November 10, 2020 Advanced, Deep Learning, Machine Learning, NLP, Project, Python, Sequence Modeling, Supervised, Text, Unstructured Data Leave a comment

How to create a poet / writer using Deep Learning (Text Generation using Python)?

Introduction From short stories to writing 50,000 word novels, machines are churning out words like never before. There are tons of examples available on the web where developers have used machine learning to write pieces of text, and the results range from the absurd to delightfully funny. Thanks to major advancements in the field of Natural Language Processing (NLP), machines are able to understand the context and spin up tales all by themselves. […]

November 4, 2020 Advanced, Algorithm, Deep Learning, NLP, Text, Unstructured Data, Unsupervised Leave a comment

Build a Natural Language Generation (NLG) System using PyTorch

Overview Introduction to Natural Language Generation (NLG) and related things- Data Preparation Training Neural Language Models Build a Natural Language Generation System using PyTorch Introduction In the last few years, Natural language processing (NLP) has seen quite a significant growth thanks to advancements in deep learning algorithms and the availability of sufficient computational power. However, feed-forward neural networks are not considered optimal for modeling a language or text. This is because the feed-forward network does not take into consideration the […]

October 29, 2020 Algorithm, Intermediate, NLP, Python, Sequence Modeling, Supervised, Text, Unstructured Data Leave a comment

OpenAI’s GPT-2: A Simple Guide to Build the World’s Most Advanced Text Generator in Python

Overview Learn how to build your own text generator in Python using OpenAI’s GPT-2 framework GPT-2 is a state-of-the-art NLP framework – a truly incredible breakthrough We will learn how it works and then implements our own text generator using GPT-2 Introduction “The world’s best economies are directly linked to a culture of encouragement and positive feedback.” Can you guess who said that? It wasn’t a President or Prime Minister. It certainly wasn’t a leading economist like Raghuram Rajan. […]

September 19, 2020 Machine Learning, TEXT GENERATION Leave a comment

UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation

Despite the success of existing referenced metrics (e.g., BLEU and MoverScore), they correlate poorly with human judgments for open-ended text generation including story or dialog generation because of the notorious one-to-many issue: there are many plausible outputs for the same input, which may differ substantially in literal or semantics from the limited number of given references. To alleviate this issue, we propose UNION, a learnable unreferenced metric for evaluating open-ended story generation, which measures the quality of a generated story […]