Deep Learning, NLP, NMT, AI, ML
This advice should be most relevant to people studying machine learning (ML) and natural language processing (NLP) as that is what I did in my PhD. Having said that, this advice is not just limited to PhD students. If you are an independent researcher, want to start a PhD in the future or simply want to learn, then you will find most of this advice applicable. Pick and choose. Everyone is different. You will have the most success if you […]
Read more
Natural language processing (NLP) research predominantly focuses on developing methods that work well for English despite the many positive benefits of working on other languages. These benefits range from an outsized societal impact to modelling a wealth of linguistic features to avoiding overfitting as well as interesting challenges for machine learning (ML). There are around 7,000 languages spoken around the world. The map above (see the interactive version at Langscape) gives an overview of languages spoken around the world, with […]
Read more
The selection of areas and methods is heavily influenced by my own interests; the selected topics are biased towards representation and transfer learning and towards natural language processing (NLP). I tried to cover the papers that I was aware of but likely missed many relevant ones—feel free to highlight them in the comments below. In all, I discuss the following highlights: Scaling up—and down Retrieval augmentation Few-shot learning Contrastive learning Evaluation beyond accuracy Practical concerns of large LMs Multilinguality Image […]
Read more
Fine-tuning a pre-trained language model (LM) has become the de facto standard for doing transfer learning in natural language processing. Over the last three years (Ruder, 2018), fine-tuning (Howard & Ruder, 2018) has superseded the use of feature extraction of pre-trained embeddings (Peters et al., 2018) while pre-trained language models are favoured over models trained on translation (McCann et al., 2018), natural language inference (Conneau et al., 2017), and other tasks due to their increased sample efficiency and performance (Zhang […]
Read moreHumans perceive the world through observing a large number of visual scenes around us and then effectively generalizing—in other words, interpreting and identifying scenes they haven’t encountered before—without heavily relying on labeled annotations for every single scene. One of the core aspirations in artificial intelligence is to develop algorithms and techniques that endow computers with a strong generalization ability to learn only from raw pixel data to make sense of the visual world, which aligns more closely with how humans […]
Read more25 Feb21 Issue #119 – Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in MT This week we have a guest post from Eva Vanmassenhove, Assistant Professor at Tilburg University, Dimitar Shterionov, Assistant Professor at Tilburg University, and Matt Gwilliam, from the University of Maryland. In Translation Studies, it is common to refer to a term called “translationese” that encapsulates a set of linguistic features commonly present in human translations as opposed to originally written texts. Researchers in the […]
Read moreThe purpose of the Association for the Advancement of Artificial Intelligence, according to its bylaws, is twofold. The first is to promote research in the area of AI, and the second is to promote the responsible use of these types of technology. The result was a 35th AAAI Conference on Artificial Intelligence (AAAI-21) schedule that broadens the possibilities of AI and is heavily reflective of a pivotal time in AI research when experts are asking bigger questions about how best to responsibly develop, deploy, and integrate the technology. Microsoft and its researchers have been pursuing
Read more
Introduction to Artificial Intelligence and Machine Learning Artificial Intelligence (AI) and its sub-field Machine Learning (ML) have taken the world by storm. From face recognition cameras, smart personal assistants to self-driven cars. We are moving towards a world enhanced by these recent upcoming technologies. It’s the most exciting time to be in this career field! The global Artificial Intelligence market is expected to grow to $400 billion by the year 2025. From Startups to big organizations, all want to join […]
Read more
Prediction intervals provide a measure of uncertainty for predictions on regression problems. For example, a 95% prediction interval indicates that 95 out of 100 times, the true value will fall between the lower and upper values of the range. This is different from a simple point prediction that might represent the center of the uncertainty interval. There are no standard techniques for calculating a prediction interval for deep learning neural networks on regression predictive modeling problems. Nevertheless, a quick and […]
Read more
Machine learning model performance often improves with dataset size for predictive modeling. This depends on the specific datasets and on the choice of model, although it often means that using more data can result in better performance and that discoveries made using smaller datasets to estimate model performance often scale to using larger datasets. The problem is the relationship is unknown for a given dataset and model, and may not exist for some datasets and models. Additionally, if such a […]
Read more