Last Updated on August 14, 2020 A validation dataset is a sample of data held back from training your model that is used to give an estimate of model skill while tuning model’s hyperparameters. The validation dataset is different from the test dataset that is also held back from the training of the model, but is instead used to give an unbiased estimate of the skill of the final tuned model when comparing or selecting between final models. There is much […]

Last Updated on May 23, 2019 The amount of data you need depends both on the complexity of your problem and on the complexity of your chosen algorithm. This is a fact, but does not help you if you are at the pointy end of a machine learning project. A common question I get asked is: How much data do I need? I cannot answer this question directly for you, or for anyone. But I can give you a handful […]

Last Updated on June 17, 2019 It can be confusing when you get started in applied machine learning. There are so many terms to use and many of the terms may not be used consistently. This is especially true if you have come from another field of study that may use some of the same terms as machine learning, but they are used differently. For example: the terms “model parameter” and “model hyperparameter.” Not having a clear definition for these […]

Machine learning experiments can take a long time. Hours, days, and even weeks in some cases. This gives you a lot of time to think and plan for additional experiments to perform. In addition, the average applied machine learning project may require tens to hundreds of discrete experiments in order to find a data preparation model and model configuration that gives good or great performance. The drawn-out nature of the experiments means that you need to carefully plan and manage […]