How to Calculate Bootstrap Confidence Intervals For Machine Learning Results in Python

Last Updated on August 14, 2020

It is important to both present the expected skill of a machine learning model a well as confidence intervals for that model skill.

Confidence intervals provide a range of model skills and a likelihood that the model skill will fall between the ranges when making predictions on new data. For example, a 95% likelihood of classification accuracy between 70% and 75%.

A robust way to calculate confidence intervals for machine learning algorithms is to use the bootstrap. This is a general technique for estimating statistics that can be used to calculate empirical confidence intervals, regardless of the distribution of skill scores (e.g. non-Gaussian)

In this post, you will discover how to use the bootstrap to calculate confidence intervals for the performance of your machine learning algorithms.

After reading this post, you will know:

  • How to estimate confidence intervals of a statistic using the bootstrap.
  • How to apply this method to evaluate machine learning algorithms.
  • How to implement the bootstrap method for estimating confidence intervals in Python.

Kick-start your project with my new book Statistics for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.

Let’s get
To finish reading, please visit source site