January 19, 2021 Artificial intelligence Leave a comment

Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillation

Under now-standard techniques, such as over-parameterization, batch-normalization, and adding residual links, “modern age” neural network training—at least for image classification tasks and many others—is usually quite stable. Using standard neural network architectures and training algorithms (typically SGD with momentum), the learned models perform consistently well, not only in terms of training accuracy but even in test accuracy, regardless of which random initialization or random data order is used during the training. For instance, if one trains the same WideResNet-28-10 architecture […]

January 19, 2021 Machine Learning

How to Develop Your First XGBoost Model in Python

Last Updated on January 19, 2021 XGBoost is an implementation of gradient boosted decision trees designed for speed and performance that is dominative competitive machine learning. In this post you will discover how you can install and create your first XGBoost model in Python. After reading this post you will know: How to install XGBoost on your system for use in Python. How to prepare data and train your first XGBoost model. How to make predictions using your XGBoost model. Kick-start […]

January 19, 2021 Machine Learning

How to Choose an Activation Function for Deep Learning

Last Updated on January 19, 2021 Activation functions are a critical part of the design of a neural network. The choice of activation function in the hidden layer will control how well the network model learns the training dataset. The choice of activation function in the output layer will define the type of predictions the model can make. As such, a careful choice of activation function must be made for each deep learning neural network project. In this tutorial, you […]

January 17, 2021 Neural Machine Translation (NMT), NMT Leave a comment

Machine Translation Weekly 65: Sequence-to-sequence models and substitution ciphers

Today, I am going to talk about a recent pre-print on sequence-to-sequence models for deciphering substitution ciphers. Doing such a thing was somewhere at the bottom of my todo list for a few years, I suggested it as a thesis topic to several master students and no one wanted to do it, so I am glad that someone finally did the experiments. The title of the preprint is Can Sequence-to-Sequence Models Crack Substitution Ciphers? and the authors are from the […]

January 16, 2021 Machine Learning

Visualization for Function Optimization in Python

Function optimization involves finding the input that results in the optimal value from an objective function. Optimization algorithms navigate the search space of input variables in order to locate the optima, and both the shape of the objective function and behavior of the algorithm in the search space are opaque on real-world problems. As such, it is common to study optimization algorithms using simple low-dimensional functions that can be easily visualized directly. Additionally, the samples in the input space of […]

January 15, 2021 NLP

ACL 2018 Highlights: Understanding Representations and Evaluation in More Challenging Settings

This post discusses highlights of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018). This post originally appeared at the AYLIEN blog. I attended the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018) in Melbourne, Australia from July 15-20, 2018 and presented three papers . It is foolhardy to try to condense an entire conference into one topic; however, in retrospect, certain themes appear particularly pronounced. In 2015 and 2016, NLP conferences were dominated […]

January 14, 2021 Python

How to Randomly Select Elements From a List in Python

Introduction Selecting a random element or value from a list is a common task – be it for randomized result from a list of recommendations or just a random prompt. In this article, we’ll take a look at how to randomly select elements from a list in Python. We’ll cover the retrieval of both singular random elements, as well as retrieving multiple elements – with and without repetition. Selecting a Random Element From Python List The most intuitive and natural […]

January 14, 2021 Python

Introduction to Data Visualization in Python with Pandas

Introduction People can rarely look at a raw data and immediately deduce a data-oriented observation like: People in stores tend to buy diapers and beer in conjunction! Or even if you as a data scientist can indeed sight read raw data, your investor or boss most likely can’t. In order for us to properly analyze our data, we need to represent it in a tangible, comprehensive way. Which is exactly why we use data visualization! The pandas library offers a […]

January 14, 2021 Artificial intelligence Leave a comment

VinVL: Advancing the state of the art for vision-language models

Humans understand the world by perceiving and fusing information from multiple channels, such as images viewed by the eyes, voices heard by the ears, and other forms of sensory input. One of the core aspirations in AI is to develop algorithms that endow computers with a similar ability: to effectively learn from multimodal data like vision-language to make sense of the world around us. For example, vision-language (VL) systems allow searching the relevant images for a text query (or vice […]

January 14, 2021 Low-Resource NMT, NLP

Issue #113 – Optimising Transformer for Low-Resource Neural Machine Translation

14 Jan21 Issue #113 – Optimising Transformer for Low-Resource Neural Machine Translation Author: Dr. Jingyi Han, Machine Translation Scientist @ Iconic Introduction The lack of parallel training data has always been a big challenge when building neural machine translation (NMT) systems. Most approaches address the low-resource issue in NMT by exploiting more parallel or comparable corpora. Recently, several studies show that instead of adding more data, optimising NMT systems could also be helpful to improve translation quality for low-resource language […]

« 1 … 741 742 743 744 745 … 974 »