Methods for Preventing Overfitting

Before we can prevent overfitting, we need to give the term a rigorous meaning. We’ve talked about a model being overfit when it matches part of the noise in a data set rather than the true underlying signal. But if we don’t know the truth, how can we tell that we’re getting further away from it rather than closer?

The trick is to make clear what we mean by truth. For our purposes, a predictive model is close to the truth when its predictions about future data are accurate. But of course we don’t have access to future data; we only have data from the past. Thankfully, we can simulate what it would be like to have access to future data by splitting up our past data into two parts.