Deep Learning, NLP, NMT, AI, ML
Articles About Natural Language Processing
News 👋 Hi there, welcome to the 12th issue of the 🤗 newsletter! Here’s what’s been brewing this month: Part 2 of the 🤗 course AutoNLP Free Tier for one week We welcome GPT-J to the 🤗 Transformers family … and more! 🎓 November 15-19: Part 2 of the 🤗 course goes live! We’re excited to release the second
Read more
Abstract Recent work in open-domain conversational agents has demonstrated that significant improvements in humanness and user preference can be achieved via massive scaling in both pre-training data and model size (Adiwardana et al., 2020; Roller et al., 2020). However, if we want to build agents with human-like abilities, we must expand beyond handling just text. A particularly important topic is the ability to see images and communicate about what is perceived. With the goal of getting humans to engage in […]
Read more
Abstract Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge. In this work we explore the use of neural-retrieval-in-the-loop architectures – recently shown to be effective in open-domain QA – for knowledge-grounded dialogue, a task that is arguably more challenging as it requires querying based on complex multi-turn dialogue context and generating conversationally coherent responses. We study various types of architectures with multiple components – retrievers, rankers, and encoder-decoders – with […]
Read more
Abstract We propose the first general-purpose gradient-based adversarial attack against transformer models. Instead of searching for a single adversarial example, we search for a distribution of adversarial examples parameterized by a continuous-valued matrix, hence enabling gradient-based optimization. We empirically demonstrate that our white-box attack attains state-of-the-art attack performance on a variety of natural language tasks, outperforming prior work in terms of adversarial success rate with matching imperceptibility as per automated and human evaluation. Furthermore, we show that a powerful black-box […]
Read more
November 7, 2021 By: Soumya Batra, Shashank Jain, Peyman Heidari, Ankit Arun, Catharine Youngs, Xintong Li, Pinar Donmez, Shawn Mei, Shiun-Zu Kuo, Vikas Bhardwaj, Anuj Kumar, Michael White Abstract We propose a novel framework to train models to classify acceptability of responses generated by natural language generation (NLG) models, improving upon existing sentence transformation and model-based approaches. An NLG response is considered acceptable if it is both semantically correct and grammatical. We don’t make use of any human references making […]
Read more
Abstract Despite rapid progress in the recent past, current speech recognition systems still require labeled training data which limits this technology to a small fraction of the languages spoken around the globe. This paper describes wav2vec-U, short for wav2vec Unsupervised, a method to train speech recognition models without any labeled data. We leverage self-supervised speech representations to segment unlabeled audio and learn a mapping from these representations to phonemes via adversarial training. The right representations are key to the success […]
Read more
The Localization Problem (LP) is a glaring dark cloud hanging over the state of affairs in applied deep learning. And acknowledging this problem, I believe, will enable us to make better use of applied AI and expand our knowledge in how the business market will form. Defining LP: There is a limit to how much large centralized language models can generalize at scale given: 1) that different users inherently have varying definitions of ground-truths due to inter-dependencies to their unique […]
Read more
David is killing it! Welcome back NLP peeps! Do you miss the old days? The old internet days of modem calling, static websites, you know… a time of innocence where developers were innovating the backbone of the internet at hyper speeds? Well, we are very much going thru that right now via the Web 3.0 revolution. Cryptocurrencies usually get all of the attention but there is something else at play and it involves the entire web. You see, the current […]
Read moreHuman error causes 23% of unplanned downtime in manufacturing. As you may know, unplanned downtime in manufacturing is a major cause of lost revenues. Can AI help reduce human errors in manufacturing? The quick answer is yes! AI can help mimic human decision-making on specific tasks. For example, on analyzing the image of a traffic stop, AI systems can be trained to detect the presence of objects such as a person, a stop sign, or a road bump. Given an […]
Read more
Abstract Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for multi-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective. The new training objective contains not only terms for predicting the subject and object of a given triple, but also a term for predicting the relation type. We analyse how this new objective impacts multi-relational learning in KBC: experiments on […]
Read more