Weight Initialization for Deep Learning Neural Networks
Weight initialization is an important design choice when developing deep learning neural network models.
Historically, weight initialization involved using small random numbers, although over the last decade, more specific heuristics have been developed that use information, such as the type of activation function that is being used and the number of inputs to the node.
These more tailored heuristics can result in more effective training of neural network models using the stochastic gradient descent optimization algorithm.
In this tutorial, you will discover how to implement weight initialization techniques for deep learning neural networks.
After completing this tutorial, you will know:
- Weight initialization is used to define the initial values for the parameters in neural network models prior to training