Polynomial Regression

 

Our hypothesis function need not be linear (a straight line) if that does not fit the data well. We can change the behavior or curve of our hypothesis function by making it a quadratic, cubic or square root function (or any other form).

For example, if our hypothesis function is hθ(x)=θ0+θ1x1h_\theta(x) = \theta_0 + \theta_1 x_1 then we can create additional features based on x1x_1, to get the quadratic function hθ(x)=θ0+θ1x1+θ2x12h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2 or the cubic function hθ(x)=θ0+θ1x1+θ2x12+θ3x13h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2 + \theta_3 x_1^3

In the cubic version, we have created new features x2x_2 and x3x_3 where x2=x12x_2 = x^2_1 and x3=x13x_3 = x^3_1.

14-polynomial-regression-1

To make it a square root function, we could do:
hθ(x)=θ0+θ1x1+θ2x1h_\theta(x) = \theta_0 + \theta_1 x_1 + \theta_2 \sqrt{x_1}

15-polynomial-regression-2

One important thing to keep in mind is, if you choose your features this way then feature scaling becomes very important. Eg. if x1x_1 has range 1 - 1000 then range of x12x^2_1 becomes 1 - 1,000,000 and that of x13x^3_1 becomes 1 - 10910^9