Introduction
Till now, We have read about Gradient Descent,Min-Batch Gradient Descent,Stochastic Gradient Descent and other type of Gradient Descents. In this post we will learn about Polynomial Regression.
What if your data is actually more complex than a simple straight line? Surprisingly,
you can actually use a linear model to fit nonlinear data. A simple way to do this is to
add powers of each feature as new features, then train a linear model on this extended
set of features. This technique is called Polynomial Regression.
Polynomial regression is a type of regression analysis in machine learning where the relationship between the independent variable (feature) and the dependent variable (target) is modeled as an nth degree polynomial. This allows the algorithm to capture non-linear relationships between the variables.
Mathematical Representation
The general form of a polynomial regression equation with one independent variable is given by:
Y=β0+β1⋅X+β2⋅X2+…+βn⋅Xn+ϵ
Here:
- Y is the dependent variable.
- X is the independent variable.
- β0,β1,β2,…,βn are the coefficients of the polynomial terms.
- n is the degree of the polynomial.
- ϵ represents the error term.
Generate Synthetic Data
Let’s look at an example. First, let’s generate some nonlinear data, based on a simple
quadratic equation.
m = 100
X = 6 * np.random.rand(m, 1) - 3
y = 0.5 * X**2 + X + 2 + np.random.randn(m, 1)
Clearly, a straight line will never fit this data properly. So let’s use Scikit-Learn’s Poly nomialFeatures class to transform our training data, adding the square (2nd-degree polynomial) of each feature in the training set as new features (in this case there is just one feature):
from sklearn.preprocessing import PolynomialFeatures
poly_features = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly_features.fit_transform(X)
X_poly now contains the original feature of X plus the square of this feature. Now you can fit a LinearRegression model to this extended training data
lin_reg = LinearRegression()
lin_reg.fit(X_poly, y)
lin_reg.intercept_, lin_reg.coef_
(array([1.78134581]), array([[0.93366893, 0.56456263]]))
Not bad: the model estimates y = 0 . 56*x1^2+0 . 93*x1+1 . 78 when in fact the original function was y = 0 . 5*x1^2+1 . 0*x1 +2 . 0 + Gaussian noise.
Note that when there are multiple features, Polynomial Regression is capable of finding relationships between features (which is something a plain Linear Regression model cannot do). This is made possible by the fact that PolynomialFeatures also adds all combinations of features up to the given degree. For example, if there were two features a and b, PolynomialFeatures with degree=3 would not only add the features a^2, a^3, b^2, and b^3, but also the combinations ab,a^2b, and ab^2.
PolynomialFeatures(degree=d) transforms an array containing n features into an array containing (n + d)!/d!*n! features, where n! is the factorial of n, equal to 1 × 2 × 3 × ⋯ × n. Beware of the combinatorial explosion of the number of features!
Important Notice for college students
If you’re a college student and have skills in programming languages, Want to earn through blogging? Mail us at geekycomail@gmail.com
For more Programming related blogs Visit Us Geekycodes . Follow us on Instagram.