What Does Overfitting Mean?
Overfitting is a condition that occurs when a machine learning or deep neural network model performs significantly better for training data than it does for new data.
Overfitting is the result of an ML model placing importance on relatively unimportant information in the training data. When an ML model has been overfit, it can't make accurate predictions about new data because it can't distinguish extraneous (noisey) data from essential data that forms a pattern.
For example, if a computer vision (CV) program's task is to capture license plates, but the training data only contains images of cars and trucks, the learning model might overfit and conclude that having four wheels is a distinguishing characteristic of license plates. When this happens, the CV programming is likely to do a good job capturing license plates on vans, but fail to capture license plates on motorcycles.
The most common causes of overfitting include the following:
- The data used to train the model is dirty and contains large amounts of noise.
- The model has a high variance with data points that are very spread out from the statistical mean and from each other.
- The size of the training dataset is too small.
- The model was created by using a subset of data that does not accurately represent the entire data set.