Finding degree

Question

How to find the right degree?

If you perform high-degree Polynomial Regression, you will likely fit the training data much better than with plain Linear Regression. Let us apply a degree-\(200\) polynomial model to the preceding training data, and compare the result with a linear model and cubic model (\(3^{rd}\)-degree polynomial).

    

Notice how the 200-degree polynomial model wiggles around to get as close as possible to the training instances. Of course, this high-degree Polynomial Regression model is overfitting the training data, while the linear model is underfitting it. The model that will generalize best, in this case, is the cubic model. It makes sense, right.!!

In general, you won’t know what function generated the data, so:

• How can you decide how complex your model should be?

• How can you tell that your model is overfitting or underfitting the data?

Solution: Learning curves