Overfitting

What Does Overfitting Mean?

Overfitting is a condition that occurs when a machine learning or deep neural network model performs significantly better for training data than it does for new data.

Overfitting is the result of an ML model placing importance on relatively unimportant information in the training data. When an ML model has been overfit, it can't make accurate predictions about new data because it can't distinguish extraneous (noisey) data from essential data that forms a pattern.

For example, if a computer vision (CV) program's task is to capture license plates, but the training data only contains images of cars and trucks, the learning model might overfit and conclude that having four wheels is a distinguishing characteristic of license plates. When this happens, the CV programming is likely to do a good job capturing license plates on vans, but fail to capture license plates on motorcycles.

The most common causes of overfitting include the following:

• The data used to train the model is dirty and contains large amounts of noise.
• The model has a high variance with data points that are very spread out from the statistical mean and from each other.
• The size of the training dataset is too small.
• The model was created by using a subset of data that does not accurately represent the entire data set.

Techopedia Explains Overfitting

When an algorithm is is either too complex or too flexible, it can end up overfitting and focus on the noise (irrelevant details) instead of the signal (the desired pattern) in training data. When an overfit model makes predictions that incorporate noise it will still perform quite well on its training data but it will perform quite poorly on new data.

There are a number of techniques that machine learning researchers can use to mitigate overfitting including:

• Cross-validation
• Regularization
• Early stopping
• Pruning
• Ensembling