This is a short post created for quick reference on techniques which could be used for model evaluation & selection and model and algorithm comparision. This would be very helpful for those aspiring data scientists beginning to learn machine learning or those with advanced data science skills as well.
The image has been taken from this blog, Comparing the performance of machine learning models and algorithms using statistical tests and nested cross-validation authored by Dr. Sebastian Raschka
The above diagram provides prescription for what needs to be done in each of the following areas with small and large dataset. Very helpful, indeed.
- Model evaluation
- Model selection
- Model and algorithm comparison using statistical hypothesis tests