Model Evaluation

What are the 3 types of model error?

1. Model bias. An error originating from erroneous assumptions in the learning algorithm. Models suffering from high bias can cause an algorithm to miss the relevant relations between features and the response variable, commonly referred to as underfitting.

2. Model variance. This is related to the instability of the model - how sensitive the model is to small fluctuations in the utilised training dataset. Models suffering from high variance can cause an algorithm to miss relevant relations, commonly referred to as overfitting. 

3. Random error. A model is a simplification of reality and can never be perfect, so it will therefore always suffer from random errors. A different way of interpreting the random error is by looking at it as a measure of the amount of data noise present.

How can you fix model error?

As random error is inevitable, the only two error components that can be minimised are bias and variance. The right path for a data scientist is to always try to find a model that has a balance between accuracy and precision, called the bias-variance trade-off. 

Fixing high bias can be achieved by adding more input features or by increasing the complexity of the model. Fixing high variance models can be achieved by using more training data and reducing the number of features so that only the most important are considered.