When to Use k-Nearest Neighbors

The k-nearest neighbors method has a few disadvantages. Making predictions is very computationally intensive because the distance to every point has to be calculated. Furthermore, in a dataset with many variables, it can be difficult to determine the appropriate weights or whether some variables should be eliminated. Optimization can help with this, but it can take a very long time to find a good solution with big datasets.

Still, as you’ve seen in this chapter, kNN offers a number of advantages over other methods. The flip side to the computational intensity of making a prediction is that new observations can be added to the data without any computational effort. It’s also easy to interpret exactly what’s happening because you know it’s using the weighted value of other observation to make its predictions.