Mini-batch gradient descent is a variant of gradient descent algorithm that is commonly used to train deep learning models. The idea behind this algorithm is to divide the training data into batches, which are then processed sequentially. In each iteration, we update the weights of all the training samples belonging to a particular batch together. This process is repeated with different batches until the whole training data has been processed. Compared to batch gradient descent, the main benefit of this […]

Optimization is a process where we try to find the best possible set of parameters for a deep learning model. Optimizers generate new parameter values and evaluate them using some criterion to determine the best option. Being an important part of neural network architecture, optimizers help in determining best weights, biases or other hyper-parameters that will result in the desired output. There are many kinds of optimizers available in PyTorch, each with its own strengths and weaknesses. These include Adagrad, […]

Training data is the set of data that a machine learning algorithm uses to learn. It is also called training set. Validation data is one of the sets of data that machine learning algorithms use to test their accuracy. To validate an algorithm’s performance is to compare its predicted output with the known ground truth in validation data. Training data is usually large and complex, while validation data is usually smaller. The more training examples there are, the better the […]

The multilinear regression model is a supervised learning algorithm that can be used to predict the target variable y given multiple input variables x. It is a linear regression problem where more than one input variables x or features are used to predict the target variable y. A typical use case of this algorithm is predicting the price of a house given its size, number of rooms, and age. In previous tutorials, we focused on simple linear regression where we […]

A neural network architecture is built with hundreds of neurons where each of them takes in multiple inputs to perform a multilinear regression operation for prediction. In the previous tutorials, we built a single output multilinear regression model that used only a forward function for prediction. In this tutorial, we’ll add optimizer to our single output multilinear regression model and perform backpropagation to reduce the loss of the model. Particularly, we’ll demonstrate: How to build a single output multilinear regression […]

While in the previous few tutorials we worked with single output multilinear regression, here we’ll explore how we can use multilinear regression for multi-target predictions. Complex neural network architectures are essentially having each neuron unit to perform linear regression independently then pass on their result to another neuron. Therefore, knowing how such regression works is useful to understand how a neural network performs multi-target predictions. The goal of this article is to provide a step-by-step guide for the implementation of […]

The multi-target multilinear regression model is a type of machine learning model that takes single or multiple features as input to make multiple predictions. In our earlier post, we discussed how to make simple predictions with multilinear regression and generate multiple outputs. Here we’ll build our model and train it on a dataset. In this post, we’ll generate a dataset and define our model with an optimizer and a loss function. Then, we’ll train our model and visualize the results […]

Logistic regression is a statistical technique for modeling the probability of an event. It is often used in machine learning for making predictions. We apply logistic regression when a categorical outcome needs to be predicted. In PyTorch, the construction of logistic regression is similar to that of linear regression. They both applied to linear inputs. But logistic regression is specifically classification problems, such as classifying into one of the two outcomes (0 or 1). In this tutorial, we’ll focus on […]

In order to build a classifier that accurately classifies the data samples and performs well on test data, you need to initialize the weights in a way that the model converges well. Usually we randomized the weights. But when we use mean square error (MSE) as loss for training a logistic regression model, we may sometimes face a few problems. Before we get into further details, note that the methodology used here also applies to classification models other than logistic […]

While a logistic regression classifier is used for binary class classification, softmax classifier is a supervised learning algorithm which is mostly used when multiple classes are involved. Softmax classifier works by assigning a probability distribution to each class. The probability distribution of the class with the highest probability is normalized to 1, and all other probabilities are scaled accordingly. Similarly, a softmax function transforms the output of neurons into a probability distribution over the classes. It has the following properties: […]