Articles About Natural Language Processing

The NLP Cypher | 10.17.21

David is killing it! Welcome back NLP peeps! Do you miss the old days? The old internet days of modem calling, static websites, you know… a time of innocence where developers were innovating the backbone of the internet at hyper speeds? Well, we are very much going thru that right now via the Web 3.0 revolution. Cryptocurrencies usually get all of the attention but there is something else at play and it involves the entire web. You see, the current […]

Read more

AI in Manufacturing: 4 Real-World Examples

Human error causes 23% of unplanned downtime in manufacturing. As you may know, unplanned downtime in manufacturing is a major cause of lost revenues. Can AI help reduce human errors in manufacturing? The quick answer is yes! AI can help mimic human decision-making on specific tasks. For example, on analyzing the image of a traffic stop, AI systems can be trained to detect the presence of objects such as a person, a stop sign, or a road bump. Given an […]

Read more

Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations

Abstract Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for multi-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective. The new training objective contains not only terms for predicting the subject and object of a given triple, but also a term for predicting the relation type. We analyse how this new objective impacts multi-relational learning in KBC: experiments on […]

Read more

Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency

August 30, 2021 By: Yangyang Shi, Varun Nagaraja, Chunyang Wu, Jay Mahadeokar, Duc Le, Rohit Prabhavalkar, Alex Xiao, Ching-Feng Yeh, Julian Chan, Christian Fuegen, Ozlem Kalinli, Michael L. Seltzer Abstract We propose a dynamic encoder transducer (DET) for on-device speech recognition. One DET model scales to multiple devices with different computation capacities without retraining or fine-tuning. To trading off accuracy and latency, DET assigns different encoders to decode different parts of an utterance. We apply and compare the layer dropout […]

Read more

Contrast and Classify: Training Robust VQA Models

Abstract Recent Visual Question Answering (VQA) models have shown impressive performance on the VQA benchmark but remain sensitive to small linguistic variations in input questions. Existing approaches address this by augmenting the dataset with question paraphrases from visual question generation models or adversarial perturbations. These approaches use the combined data to learn an answer classifier by minimizing the standard cross-entropy loss. To more effectively leverage augmented data, we build on the recent success in contrastive learning. We propose a novel […]

Read more

The NLP Cypher | 10.03.21

RAFT is a few-shot classification benchmark that tests language models: – across multiple domains (lit reviews, medical data, tweets, customer interaction, etc.) – on economically valuable classification tasks (someone inherently cares about the task) – with evaluation that mirrors deployment (50 labeled examples per task, info retrieval allowed, hidden test set)  

Read more

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

Abstract We propose using self-supervised discrete representations for the task of speech resynthesis. To generate disentangled representation, we separately extract low-bitrate representations for speech content, prosodic information, and speaker identity. This allows to synthesize speech in a controllable manner. We analyze various state-of-the-art, self-supervised representation learning methods and shed light on the advantages of each method while considering reconstruction quality and disentanglement properties. Specifically, we evaluate the F0 reconstruction, speaker identification performance (for both resynthesis and voice conversion), recordings’ intelligibility, […]

Read more

Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications

November 7, 2021 By: Shuo Sun, Ahmed El-Kishky, Vishrav Chaudhary, James Cross, Francisco Guzmán, Lucia Specia Abstract Sentence-level Quality Estimation (QE) of machine translation is traditionally formulated as a regression task, and the performance of QE models is typically measured by Pearson correlation with human labels. Recent QE models have achieved previously-unseen levels of correlation with human judgments, but they rely on large multilingual contextualized language models that are computationally expensive and thus infeasible for many real-world applications. In this […]

Read more
1 2 3 67