How to Choose an Optimization Algorithm
Optimization is the problem of finding a set of inputs to an objective function that results in a maximum or minimum function evaluation.
It is the challenging problem that underlies many machine learning algorithms, from fitting logistic regression models to training artificial neural networks.
There are perhaps hundreds of popular optimization algorithms, and perhaps tens of algorithms to choose from in popular scientific code libraries. This can make it challenging to know which algorithms to consider for a given optimization problem.
In this tutorial, you will discover a guided tour of different optimization algorithms.
After completing this tutorial, you will know:
- Optimization algorithms may be grouped into those that use derivatives and those that do not.
- Classical algorithms use